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, , * - ' Abstract > . , 

* • • • 

Tbis report' is intended to serve as a resource for the 
development of ■ management and instructional guidelines ' for 
computer-based education (CBE) . Although the data in it 
Were gathered from, PLATO projects only, they represent pro-, 
jects w,hich varied^ widely in tstrget populations ( ej.ementar 
through; professional students) subject matter content, type 
of imiplementat ion and siz.e and scope. 'Therefeire, it is 
expected that thfe report wiM be- useful to deyelopers of CBE 
in general, antl not only' t^ PLATO us^rs. 

Critical incident^ are. defined in terms of four cri- 
teria. > Tl^en more than 125 case histories of' critical inci- 
dents are documented.' They ar« organized by\topics, rather 
than projects, which in effect will serve as a tgxonomy of, 
matters or issues which' are criticaj. during project develop- 
ment. The report also includes summaries and analyses of 
the , processes and procedures and ,their subsequent effects. 

Just a few conclusions Will be^ given here., CBE *was 
implemented most ^nioothly when there was a conscious effort 
to develop good relations .with instructors and administra- 
t^ors. Successful projects were tho^^e which had initial- 
'pla-ns for such matters^ as pro ject . procedure's , organization, 
pbjectiyes and - evaluation . Directors who* planned* contin- 
gencies, in case of failure, to meet goals', increased the 
probability of the ecrtiveness of their pro jects ''Evalua- 
tion* was essential ,• throughout lesion development as well as 
after lesson completion. Many models of staff org-anization 
were noted,, but there was no single best morfe4 fdf^lesson 
development* Successful staff members were those 'whose 
expertise was not limited to a sfngTe area, such as pro- 
gramming, but who had breadth in 'many areas including 
rte^ching in an i^tecactive environment. / \^ 
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,. , . _ .Chapter 1 • . • f " 

/-. * - ■ ^ _ I^JTKODUCTION ' , 

^ A flaclcgrouhd and Rationale-- 

' - PLATO is a. aci'at^ely new technology (cf. Lyi^an,. '1975) , 
as is all computer-based' education .(CBE). When* PLATO III, 
the first classroom-oriented version of ,the •system, was 
introduced, the major educational efforts- were exploratory 
and evolutionary. Directors of.^ these early pnsijects had to. 
f ormulate ' plans without the t)eriefit^f 6ther people's 
experience. This wa« duetto the fact that CBE, and- the ^ • 
, PLATO III system in particular, was a novel , and unique 
medium. Previous^ research in related cfreas. such as ^ 
prpgrammed fpstruction and curriculum developmerft was an ' • 
inadequate source of Infornfa tioa. Project directors tended 
to try out ideas in order to learn* from experience. . 
Sometimes, they shared the knowledge ga^ined with oth^r " 
projects, Fo> the most 'part, 'this information was not 
recorded . . ' > , • ' * 

With the advent of t;he .PI^ATO IV-^stem a few years , 
later, the system's capability expanded to handling hundreds 
,of terminals simultaneously. Directors of new and larger 
^projects had to make decisions not only, about instructional 
design,'^ but also" /about project management and organization. 
The new dimensions or CBE made it necessary to revise old 
concepts and sometimes even* .to develop new procedures. Di- 
rec^tors' could not anticipate all aspects of planning that, 
would be nec4ssary, and there were scant resources to guide 
them. Unlike other CBE projects (cf. Faust, 1974) eabh 
group using PLATO made its *own decisi-ons about organization, 
procedures, staff selection, lesson de9ign, and evaluation. 
The groups varied widely in size, setting, and ediicatronal . 
level. AfS knowiletlge and 'expfrieftce accumulated, stafjf f ^' 
members.^ of each" group made modifications or even complete 
revisions of- one or more aspects of their work^' ^ ^ 
Sometimes,* groups who were working under different . , 
circumstances came to similar conclusions at>p^ut ef/ective. 
ways to reorganize or Ijo improve some aspect of their ^ * 
project. * For example, two projects that were very different 
in, size, scope, and target population found that tfie same ^ 
organizational structure .met^, their needs.. Sometimes, hdw- 
ever, when confrojited with similar^ decisions , t|ie various 
groups chose different alternatives because' of their partix:- 
ular iCi'rcumstances, For example, someJ groups ilec Id ed to 
adapt* available curricular m'ater^iadLs to the' CBE ^system , 
whereas others devel^oped new curricula, ^What was ^a good 
idea for project was not necessarily considered tq ^e a 

good tdeaffor another. 




^Tti^^ sum total of thig experience can' provide a valua,ble 
data'^base^ for guiding t^e d^V^lQpment of many aspects gS ^' 
future'*CBE pro'jects. Specific documentation of'circum- 
stsn-c^es- preceding '^n*d surrounding tlTese incidents and deci- 
sions is essential *so trhat'*the consequences *may be»unci*er- 
st6od in proper context. -This, kind of inf^orra^tion has not^ 
b.een generally available. Although, most projects have . 
provided r^por.ts' and summaries of their gtctivities, t^hese 
• h"av^ varied considerabl 'They include such materials as 
'.catalogs of 1 esspns , 'rfata on lesson usage s^urrimaries of 
pt-u^^l^nt per fbrmaince , ^nd compar^isan to cpntrol groups. Most? 
are 'summative evaJ-uation^^ giving little or no formabive 
eyaluation datd (i.e., information abojut the ^evolution of ' 
projecti processes and procedtires) • Those reports* which do 
include s'6me' of this in form^ation- tend to be irvcomplete. 
They are prope to^overTook -and omit decisions and events 
which produced negative re*sults. .In some situations, where 
there wa^ a complete turnover" of personnel, the report 
writers t^nd .to irrclud^ little or no -info^f^at^ori^ about the 
earlier phase of the project. 

Oulside evaluators have written about^ some aspects of 
project develgpmenb; ^but .these are limited to thfir' prarti- 
Qular focus\ House ( 1 973) provided ^tensive process docu- 
.mentation aboutkone implementation^ effort but neg^^arily 
. discussed ' events tJiat served .to illuminate* the author'sj 
topic,' the pol itics' of' innovation' tn education", \Simil€irly, 
Mahler and his colleagues p97^> documentecj PLATO courseware* 
development, but tJ;ie ^ inTormation is general an<J lacks ^e 
detail that would* be ' needecl^ for de^veloping guideTines^ for 
the f utyre . ' ' . ► r • 

3^ ' ' # B. Purpose ' . 

• The purpose of this 'neport is rb. provide *a -broad-based 
set o'f case histories arrd <lecisians that were observed , to 
hpve specific ^ff^cts br lo-ng-term impact on t?he smoothness 
of project ^implementation. These incidents are intended *tq ' 
proVid* a data base for developing guidelines for futurevCDE 
projects in g^hewal arid^ P^LATO^ pVoJects in partiCAilar. It is 
also intended to^ serve as a first apprS^'Ny tion to ^ taxon-, 
omy of factors thai; sfiould be considered in project manage- 
ment an.d instructional design. ^ :* • 

^ • * . C. Method : 

1,. Resources < * ^ --ft* 
* • • • * 

Three sources of infarmation were used: person'al 
accounts 'of the individuals who .dev.eloped lessQns gitd were 
directly involved in; the projects or were in a position to 
mak,e perso'nal.- observations ; interviews by the editor with 



some directors and, staff members of various projects; and 
published documeptis such as^ those previously l isted . < 

2r; Definitions Snd' criteria for iHclusion of cases 

anagan {\9l5M) developed atechnique tor arriving at 
Jpb specifications by documenting and ■ classifying critical ' 
Incidents, tie defined an incident as critical if its purr' 
pose was clear and the results^ wenfe "^definite enough' to 
"leave little 'doubf about .its -^f f ects . " This definition* ha« 
been adapted here ^ fo^ evaluating aspects* of CBE projects and 
has served as a star*ting point fof'the inclusion -'of inci- 
d^ntsjin this reports Three additional definitions were 
included to cover other kinds of situations. Thus there 
were four criteria for^nclusio^ of incidents. . ^ 

• X)\ k decision or inoidenti .wais^ .considered cri1|4car ^ 
•ir*the circulnstances surrounding it' were clear • 
'and the resif'lts or effects were definite. 

?) There were effects or results ,oTP sittuatrons that 
could be traced to. the fact^that a^decision was 
not made. Theref oVe,, »f ^.r sorae incidents included 
hire the effecW were -tracefd 'to the failure 'of a . 
person in a. posit ion. pf responsibility to make _ 
, a particular decision. , r ' 

3) Therq wer^*- situations in which a decision had to 
. .'be made to choose between alternatives.' The • ' 

observed or reported' advantages^ and disadvantages 
of the alternative^provided valuable' information. 

4) In sortie '^nstandes , a * sruccessf ul- scheme or*, decision 
evol^^d-^thi^ough' repeated iterations of situation- ^ 
decision-;effect cycles. Since these situations, met 
the ultimate objective of this repartt,.they were 
included . - . ^ / 

3 ; Pr(3:e^edures 

The initial thrust was an informal "brain/ storrriing" 
session>f.in which ttae authors reported incident/s which came 
to mfnd* As each individual recalled events "about partic-/ 
ulap aspects of project. deVelB^pment the memory of'otl^s * 
was stimulated ^ and ^activated. ' This ^ir^itial set of events 
was then organized into^ what seemed like a reasonable set of^ 
topic lieadings. Subsequently , the authors held meetings to, 
adi more information and *help 'clarify tfte , definitions and 
criteria-for including inci<hents^ in this report.. The editor 
4-nterviewed 12 additional CBE staff members representing 9 



d-ifferent projects, '^'nd/revi?\i*ed* mojr^e than 20 project papers 
and reports for appi*'6'pnlate , inform^tiion . ' , . 

*i In 'most pro jectsv^-^py-i^lral'sO^rAT^ resulted, . 

ofte,n in Various aspec ts 'x^r- i^^t-Kuc txon an'd <nanagejien.t . 
•Since the c^ses" 6ite? bWow were classified according to* 
topics, rather'than p^ojects, it was necessary to repeat the 
c ircumsfartces in some case'^ in order to make the, report mdr^ 
readable-' ' » * / / - 

Credibility \ / ^ \ ' . ' " 

♦ It should be noted th§t the incidentrs reported below* ' 
may^be somewhat biased:. Memory tends tb' be selective;, 
'people tend • to' remember th,o drai?latic^ and perhaps' fail to 
Recall the ordinary.. They may be incl ined. -tio recall nega-, 
tiye incidents more frequently than positivei In fact, 
recall majt- sometimes be somewhat^nc|^urate . We have tried 
to minimize this' problem by ob^ a i\nin^ reaper ts of . an incident 
from more-'than ojie source >where pos^ibl'e. - 

The^role of 'an ' individual within a pro ject^ ►also preju-i . 
dices the incidents ?w*-recalls <cf * Flanagan*, T954 ) . Itk 
order to keep- this bias to a minimum, t'he peopl^ /selected "to 
l>e interviewed wer^ chosen to represent different leyet*s of 
responsibility ar)d duties. The authors of this report- also 
represent* many projects ^nd roles within them. It is th^reJ 
Tqre hoped that ttiis-kind of selection- bias is* at a mifiimuip. 

The experiences reported hereare not intended/ to be 
used as a final authority for' judgin-g the\ adequacy of. CBE^ 
projects.^ In some\instancea the evidence is scanty and 
.inadequate for makiffg^ general izat ions The intent is'to < 
.provide tentative and preWminary^J/hfprmatiort that ca^n be' ^ 
useful as a guide, for the plfinntng and management of .^11 ' 
^acet^s of new projects.. ^ -j * . ' 

Some of the. "lessons learned^' wij^l- appear * to* be no more 
than common sense. ,But what is obvious in ^hindsight wa^ not 
so clear or even visible in foresight. The purpose* of 
reporting failures, in addition ^ suc<?e53es,^ is no^tj t^ • * 
criticize but rather to emphasize the impact of. the various 
factors in establishing apd' managing a CBE site. , 



Qfiapter - 2 , . • 
PROJECT* ESTABLISHMENT . ' 

*. • 

A. Introducing CB£- . ■ 

' ' ■ / ' ■ ' 

• Introducing CBE involved special problems ./above and ^ 

beyo-nd tho^e^ usually cdnaect^ed withf instructional -iVim^J^^ 

tion. The new mecUum,-the computer, was still f^'T'^'Ofl^Hr 

recent invention, \lost peopre had hot workedVirecSlBIWtlV 

computers -and were not yet co'rtifortable with them. Some felt 
^threatened by them or were afraid of being replaced, "dthers 
expected the hew medium to IJe a g-reart ci^re-all for* many' of 
their problems. , Another unusual" circumstajice was th«t " 
lessons wA"e oXteii used for ' instrudtiori 'as soon" as they Ve re 
finished so that instructors, were u'nabl? to famiiiariz* 
themselves with the lesson. bo^tent' and.' format before their 
students becan to us.e them Cunless* the instructors" were 
themselves Tlse authors of the lessons).' • , 

The way. in whi<;h ,the ti^ system.- was , introduced at an 
institution affected staff as.wellvas stiicl^pts. -The extent 
and the, nature of public relations and orilntalion ^}.ad ^ 
decided effect oa^fehe' acceptance, coop^ratlion , and expecta- 
tions' of^ those asso<ii(B\ed with tlje project. The. irfcidehts.' 
cited below show, three^factors that eng:endered good will 
towar^^ CBE' and inst'rtjctor acceptance an« cooperation. . ^ 
Broadly, caterer iz.^d , - they are : ( X) involvement '(.2). fal^il- ' 
iarity with CBE antl/prj, the CBE less«>and { 3) " vbluYiteersva? 
instructors, desirable ' attitudes resulted when a "deliberate 
effort ..was made «to pj-ov^ide enough time jTor^ the inAructor to 
work on the 'CBE system or to. take some meaningful • par t in 
the project, like criti-quing lessons or assuming .part of fhe 
computer' s .responsibility for , i^str^uctional management. 
Conversely^ other oases indicate that, when teachers- were not 
adequately famiaiar with the CBE . system pr, were Insuffi- ' ■ 
cie'ntly invol\;.ed,%hey resisted using' the CBE lessons and 
did not make^an effort to Integrate them intCtheir regular 
classroom- instruction . * • 

fexpectatiorts coyld not always be* met as anticipated. 
In one-sueh instance the project director planned some 
alternatives i* jj.rojeqt procedures which enabled the project 
to progf-ess even ^tho.ugh fewer;" terminals were available' than 
had. been anticipated. In other .instan'ces*, failure of the ' 
director Zo make cdntingency plans or to furnis.h explana- 
tions Xor urrfulfilled promises led to misunderstandings- and ' 
bad feelings. * . 



1 .( 'Instructor acceptance and" ^cooperation 

Casie ^l-^ - Volunteers as cooperative instructors ^ 

jl^ CBE terminals .were to, be placed in*8 prespec if ied - 
mmher of classes/within a/Iarge institution. , The project 

'director had *to ciecide on"a plan for ' selecting the limited 
number of instriictor s' who would have terminals in their 

;classroo|s. He "decided to ask f.or volunteers. The instruc- 
tors -proved to be\cooperat iv^e throughout the project in 

'spite of errors in leasons or other probletns that arose. 
Coope»ation was good) even among thpse teachers who kniew that 
terminals would riot'be available* to them the fol^lowirtg* year*. 

Case 2A1.2 - Developing good .will- among instructors' 

At one »ite, as an' lexpenimlent , ^ an entire course was 
being rewritten to become , self-paced .; (Parallel CBE and 
rton-CBE portiohs of the course j^eris , to be* developed simul- . 
taheously.) It was an intricate patcnt^oVk of trtater ials froI?^ 
many me^dia. The CBE project director had taught this course 
and pre^mbably knew the course instru(*tor s • Early in thfe 
project, Tie decided ^that he and his staff, should work very 
hard to e3'tablij5h"good relationships witj»^ the instructors 
'and' developers of non-CBE/ materials in-order to keep a 
natur^al animosi^ty from developing. He lent the other group, 
sta/f , cav?, them all siOTons, taught them to'play^CBE gamgs, 
reviewed l.^ss^ns between project and traditional staff, arid^^ 
paid, what any unknowing Outsider would deem "unnecessary" 
attention to detalls^hat reduced 'jealoulsy between groups < 
He even went sp--f5r as to halt lesson polishing wben CBE 
lessons reaoh^ a level vhere they, were -clearly superior^ to . 
the conventional mater ials .* To have polished them morer 
would not only have needlessly added to development time biit 
also might have fostered feelings- of jealousy in the devel- 
operJ of the materials using conventional media. The result 
was acceptance by the .conventijonal 'staff, and relatively ^ 
s.mooth implementation. ^ . * ' 

Case 2A1.3 - Giving ins,tructors responsibilities 

In one large • curriculum project the director decided. to 
put curriculum management under computer control. That is, 
the computer kept a complete record for each studen-t and on 
that basis decided which lesson to present next.'' The 
instructors were unfamiliar w'ith the lessoYis and with the 
hardjtfare. Although they had the option of overriding the 
computer and setting up their own sequence, they rarely 
exercised it. Th^y viewed the CBE materials as a separate 
entity ^nd not an integral paVt of the regular curriculum. 
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• • 

Epilog , Irwsubsequent yea)js, the* dec ifeion. was made to 
abandon, computer managem'ent and to'gi^ve that .respons ibil i ty * 
to the instructors. By that time they were more familiar 
with'^th the lesson5 and the system.. They had to- dp tnuch 
more workv'^iagnosing and 'pnesoriblng lessons, for each 
s^ude^t. However, it gave them a sense-'of power and control 
over the^ machine, and they had more students do more lessons 
than in previous years. The students, 'in turn, were more'. 
• intere^ted^ because the lessbns were more apprqpriate and the 
Xteacher showed a vital i/n teres t and -.participated actively . 

Case 2AV.,4 - Askrng^ instructors to cVit^ique lessons, 

' • - ' ' * 

One gro\jp of authors wanted to* evaluate their mate?ials 

with .students from the target population.^ They decided that 

the instructors ;would be more inclinfed' to cooperate ''if they 

were irtvo,lved in a meani^gf ul- way , so they afeked the 

instructors to review and t^ratique the lesson^. *At tffat^ 

tim^t the lesson's i*#re still in a' primitive state and had ^not 

been carefully pretested^. Nevertheless, thr*ee of the 

instructors were positive in their reviews and used .^BE 

lessons with one or more of their classe^..->^One instructor 

wrote ^ negative .critique and decided not to use CBE at lll.^ 

2. Instructor resistance ' \^ 

Case 3'A2.1 - Insufficient "hands on" experience* 

In the first phase of the development of a large 
curriculum project, lessons were developed -by the CBE staff 
who'were not part of the .in stitut ion that would use them. 
The decision about the educational level at ^hich to»write 
was made exclusively by the CBE staffs They afso^ decided on 
Tesson* content . Instructors who' would use these materials^ ^ 
were solicited for some advice about les*son content and 
general demonstration's were • given ,- but fto provision was made 
for "hands on" experience with the materials*. This wa5 
partly dae to the f&ct there was neither lead time nor a 
backlog of lessons; lessons CJ^re written and used immedi- 
ately. Also systems limitations mac^lt difficult to'obtain 
time for instructors to try out the lesions One 'further 
obstacle w^s that teachers had to go, to a special room to 
use the termlnaLs; As a nesglt, most of the instructors • 
tr>e^ted .the CBE 'experience as a supplementary activity and 
not as an integrated or important part of the curriculum. 
.They wanted t6 have little to do with it and* resisted * 
becoming involved. ' » • . . 

gpil'Og . When a special time was later set aside * foW them tC 
^ome tq'try out the materials, only 1 out of 25 instructors' 



showed||P even yjough time fjpr i^erusal was ^ scheduled for • 
iJieir wnv^eaience. '] * ' • 

Case,2A2.2 - Insufficient farvilfarity with, lessons 

'Lessons in a given^ subject were, deveToped and validated^ 
with a particular^ subject i3opula"tion an-d wer^ showYi to be 
effective. The l^'essons Were ^ theij of fered to another in?ti- 
tution which ha^^a similar ^taden t ,bx>cly . The institution 
did ^arra^nge for reyiev time^for gny interested instructor/ 
'^he department chairman (.who- was. very pdsitive toward the 
material) scheduled e'n6ugfi^^\irfle^ so ^hat members of/every 
class Qoul<^ complete' all appropriate m»aterials. However 
because instructors and authors ^of the pfiaterials were ^usy, 
no formal attempt was made to^ insure tha^t instructor^ at the 
r\ew sit>e were aware- of the d^tail'ed ' con tent of the 
^materials'. / \ . --'^ J , 

— 1|ery few-s^f" -the -instructors took the time to examine 
the materials. Virtually no attempt was made to coordinate- 
class instruction wilh the nraterialrs. ' Most^ instructors 
simply, sent their students to the^ CBE terrrinals at the. 
appointed time as an independent. activity . Students often 
got instruction jHr^ the CBE system for .which thsy had no * 
class prepar^t/ion , 'br instruction that -duplicated earlier 
class present/ations. Student^ attitudes were extremely , hos- 
tile since they saw- the PLATO, lessons , as waste of' their 
time. Instructor attitujdes were only-^-slightJy bet.ter. 

Epilog . Follpwing the abave experienc^,^ an effont was made 
to involve instructors in the, FevisTbn ^of the original 
materials. Although only fniY>or revistops w§re in fact 
carried out, most of the itrjstructors viewed al-1 materials 
and participated in making up a 'written guide fcrr^ tHe 
coordination ^of the material^' with classroom activities. 
^ Subsequent student and 'instructor attitudes toward CBE and 
rthe materials were above average for the institution, 
although still lower than those at t^he site that first' • 
developed the ^materials . * * 

, . * • - * * ' 

Case' 2A2.3 - Insufficient fagiiliar i ty wijh CPE \ , 

Lessons were 'developed -at a curriculum center for use 
at another institution. Since lessons wer^ us^ed by students 
just as soon as-they we*re c*oinpietetj instructor s* did not 
have a chance to S6e thetn ,f ir s , and were ,^ frequently too 
busy to make tirae to v^ew 'them at- all.. Threy were uocomfort- 
ablerwith the CBF system 'and" unfamiliar with the lesson 
c o n te n t . • ' ^\ 



i 
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• Epilog > Before the siexjond y^sr begap, the project director 
made time available for the i'nstructors to have. »\hands on" > 
experience before c1^s?qs started. They felt^ more at-ease * 
with the lerssons and this attitude was reflected in the 
•students, l^nstpuctofs alsolwere able to'hel-p studelSts who 
were ha*vin^ difficulties. 

Case-2A2.W- Schism between, staffs ' > 

%K / . 

b^e site the staff felt tha:t.th.e entire curriculum 
heeded to be made more relevant for the students and decided / 
it should be revised;.^ At. about th6 same time^ funds, werie 
made available to get seme* CBE terminals. Without further ' 
cctnsultijng the course instructors, the course director 
initiated a project to develop an entirely inew durlriculum ^ 
whijr^simultitieou^ly initiating the first lise of CBE at the 
.institution. ^ ^ ' ^ 

He hired staff members to. write the new curricula, 
10 of them new people. The plan was to hav€ tfre , ^tradi- 
tional" staff adapt the content of their courses vto the 
needs of the CBE research project. They were expected to 
.isolate all of their lectures that dealt with a specified 
. topic ; then /the content was to be given to the ' CBE ■ staff for 
les5Q^writing. ' The traditional instructors Were -expected / 
to teach the selected topics at a particular time so that 
while softe of the students attended the lectures, others 
simultaneously ^earned the same material via CBE. This 
entailed a considerable , amount of worl« for traditional 
course instructors and they did not perceive CBE as a 
benefit either to ahe , course or themselves. : FurtheVmorV 
they sav/^the CBfi project as research that wquld never be 
implemented so they could just 'wait it out. They, had no 
niofe-ivation to cooperate and were slow in* turning ov'er lesson 
materials for the CBE authors to use. They did^^not dome to - 
the' terminals, ^either to try the lessons, to ob/derve the 
students or evgn just, to play on-line games. / * 

A further, .source of fHfcition was the fact that. the . 
traditional staff had feceifed preferential -treatment over - 
the other staff at the school. Because ttie traditional ' .^^ 
staff members were developing original study programs, they . 
were exempt from some of the duties* that instructors in 
pther parts of the schoGl were expected to perf;orm. They . 
were somewhat ego involved and" rhaintained a "hands of f" .'^ ' 
policy toward' their lessons. They were hardworking, but- 
often coTi^idered arrogant by other irrstructors : As- a 
r'esult, they were disinterested and prof essidng^y ^onde- \' 
scending. No orte served as an inter face ^ betw€fen the two 
atafff. -To further complicate the matter/ ch|nnel3 of 
commidnication we^.e inadequate between, the leaders of- the two. 
groups 'and between lead'ers and their staffs. • 



Epilog. At Sbout the time this -project "was ending-, the CDE 
administrator approached staff in other 'cotirses and offered 
them use of the^ CBE : f at?illties . In this instance* in con- 
trSst to the earlier^ohe, 'CP£ was offered as a means of - 
meetings the instru(5tors ' pe4pceived ^oeeds . v Jhey were moti- 
vated to use th^ CBE system Ho. solve theTir training 
piroblems . * . ' - 

3. ExVectations an(^ - cons.eqjuent misunc^erstandings . 
Case . - Who can use ' terlninals 



Staff merrHDers of one oVganizatj^on were ' notified .that' 
they,wer.e to, receive a classroom of termj.nals. They were'not 
sure that they could fU^ly utilize the CBE terminals. 
Rather than waiting unti^l the uncertainty- wa5 resolved, they 
sprfead tbe^ word that th^ terminals would be coming ^nd 
invited suggestions a/5out thpir use in. other departments^ 
Lat^r they found that there was no excess of terminal avail- 
ability', s'q they reVieg^d and^ restricted usage. The result 
v^as' a^ago'nism 'throughout' the organisation, anxl. members of • 
o.ther ef?partmej^ts\were reluctant to act as lesson revievers 
;*or to, p^rbvide ^udertts -.for lesson trials. 

^ i ' ' - ' — ^ • ^ 

Case 2A3.2 - Fewer terminals than^-promised, 

, .At ope sijie,,a given number of teijminals were pj-omised . 

,DU# to a. manufacturer ' s. delay/ .fewer were, dei iv'er^cl ^Lni- 
tially* than anticipite-a. Conjseqently. the ratio^of authors 
to terminals* Was abpul? 3 to 1. Sinc.« the PLATO terminals' 
werfe^available 22 hours a <lay;, the staff decided to resolve 
the profeiem bji^ working in three -split shifts^ Higher pay 
was^provided as an irtcei^tive ^or night work^-"^ The authprs 
clarmpd* tha:^* this system ^reduc^d tensions. But the admlni- 
gtrators ',said "that extn/'effort was needed to coordlpate , 

.staff.' They ' neport^ some problems since the staff membefs 
were not always under- supervision, ^nd production standards 
were not Wfell .defined ; Mor.eover, the split, shifts* Appar- 
ently fostered.^cliquishness and division aqfiCng.the authors.^ 

Case 2A3".3 Hot as-^reliable a3 anticipated. 

One institution decided to get terminjal s . because vali- 
•d-ated curriculum materWl • was already available in ar^ area 
.they really ne.ecied . Instructors were disapp^olnted , however, 
.because the system^w.as^ not* as-.rdlia^le as had; Been antici-,"^ 
, pated . Administrators decided, they had , be§ji njs)ed" by th/ 

developers for test ing .hardware the developer/ sent 
'' personnel to help sol ve . problems , and in-service a/ithor 
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•tracing -for instructo^.s was prolhised. This ;i coup-led with 
[ "the- fact ".that- extra fuRding .would be^vailable i-f the CBp ' * 
system was- used, led t^^-lm proved Wtitu<je on the part of 
the ffdministtators ^asTwellTas the 'teaih'in'g staff.. , 

'Case -2A3.4 - For whom CBE is useful . * ^' 

The president, of one institution saw .u?e: of th.^ CB'E- > 
"s'ystem as a chancy to' get outside 'fund ing .anil* to reclu'ce ^. 
dependency on tenured tisach'ers, many^ of whom ibelo'pged ' to 
uni6n'S4. Therefor^,/ h< 5[fe<?id'ed to flssi^n non-^t^h.yred - •. 
tegchers the task. of t(ecQm*ing CBE.*authors../ It was -a-lso ttie^ 
c^se that these ^.ln{livicfual^5 w°ere/if>. charge c^- reme"du;aiton 
'for b'pen-admisSionS...stu.dents.- 'As aV^'sult'^ ro^ructibn- via 
CBE- Wa5 perceived, by "%&flured s-taff '.as '.^ajch'^ap -trgatmert^or 
the poorer , students and- as .'a\means'" of J-e^ucing th^e p(iwer< of * 
the union bjr c^jtting "back on." the ;be*ed for" tenured ,t€%cyhers ; 
They 'also su^g^ectec^ that the CBp/system' was' used. ^onPy' for ^ 
minorities arid, ndt a£ • prestigious -uni v.i|;s^ties . ' Yersoris-tr 
assigned to aid", in establishi^g^he • si.te . w^re-,uhfami>far^/ 
with CBE. and ^qiis.se(i .the\ ppp^V^nitj/ to. eorrefct these Urppre?- 
sLons. (CBE ^was,, in_^ fac^' , ug'^d , by prestigious universities , \ 
and this pa-r't-icular inipltementatio-n w.as ah .al^'tempt l|o 'bring 
advanced' t^hrioiogy to" schoqls with ifewf^V/resoyrces . >-*Bat ' 
begause of ^thi's faJLse impression anxi the 'pqiliay -tij -tising* (?BE 
solely for. remediation,-^ senior staff 'and member s." "of -prestige 
departmenys avpided us.ing-QBE. Students assign«lj to use'^CBE 
became identified *'a^ ' "s-loV" o^^^^ "dumb"-.' 

. • * ' ' r^. M • ■■ ' » • " • - ' • 

• •• Ch6,ice 6f JFj-oyjeat Director '' ; •' ° 

PLATa. .projects .variVd, wid'^iy-.fn goai^'^. scb^e*; ^n.d rate 
of development. Conseque.ntly directdr§. were chosen und^r 
cjiffer^nt "circumstances . .Some. We're iridivllya'l professors 
who initiated . an-d developed ^the prdjiexJt oil th'eir "own in'i'ti^- 
tive. They learned' the f UTOR-pi»pgrafnmins^'. 'Jan^u^ge , ' duthared 
lessons ancT generall-y explored t-he pp.t;'e4iti.0^_of thV.-p.IiATO 
sVst^pi. A'fter bh.ey .gai,neel, .ex'p^i&nce' lii many facets of I - 
lesgqn developme-nt, they-. expanded their\staf f 'but continued 
to be actia^el.y involYed. Other .projects were intended tb ' be 
d-^monstrat^ions of use of the PLATO s:/Stem,. In'-some .L 
instances the director Was .chosen from araong-tTie instructor*- 
aqthacs who would use .the "oiateriars; in oliher iftstances^ kh 
overall , pro ject '.director* was appointed to manage djlvers^f 
projects withip a given iftstitution . ■Su9h "a director wa 
not alwajcs^yail^ble at the time a PLATO project was" est 
blis-hed. ^ might b?.- Expected',, overall "lesson produced! 
was impeded and sl^wed^^n pr*oJects whic'^''werfe- initiated 
withoilt a .leadec or„.wit;h S'tem'porary one... 



. ^ Evfin when such iTir^ctor -V/as aTailabl,e,* he. somertimes 
iaqked the essential exper^tl^e. CBE was so new t^^ there , 
was no pool of ava^ilable ^ leader ship 'in the field pe)^ se . ^ 
ConsequenLtly. the new direptors were exper'ts in* perhaps One ' 
relevant fieW -sudh as subject cootefit, -curriculum dev.elop- 
ment, computer 'programfhg , or -admi^iistration . As it tyrned* 
out, the most successful Leaders w^re those who w^re cJonternt 
experts or curriculum c^evelopeVs artd \who took the time to 
learn Bbout in'slruct ional design and ^11 about PL'ATO. 

. In some instances the director ^expected , p'r was ^ 
exjjected to /'carry on -spme other responsibilities in 'addition 
Jt) managing I the project.,. Such individual^^ did not have time^ 
€o become acT^quately knpwl^edgeable -about tKe ^workings of the- 
PLATO system. Not only did they not have 'first-hand PLATO 
ex-perience tbemsel ves.-, they did not spend . en04jjg,h time 
observing 'the production process. * Therefore, when mariage- 
ment and policy decisions ^[lad* to be made^ they were neces- 
sarily based- ort second-hand, or sofnet'imes • conflicting 
reports, .'f^any of these decisTons were ill-advised and 
resulted in staff dissension'^and .fragmentation . One direc- ^ 
•tor -^appointed* a coordinator:^,^ alleviate the problem. It 
was . rt6t;^Gcessf ul because the coord i-nator was ^not gi*ven th^- 
author^i^ he ne^cfed to car ry^ o^ut -his resporfsibili.ties 

♦ * ' t ^ ' ' ' 

'1 . Avaiiabil irty * ^ . 

Case 2B1.1 - *Leaderless prQject * ' . 

■A newvCBE project was established but a director could 
'TOt be found. * The individual who was. responsible for 
funding, but not for projecfe putcomes, decided thaCt^it would 
be better to l^et someirhing gbing, rather than wait for '.av 
qualified le'ader. * Consequentl j( he hired staff members*, some 
of them rejects 'from -other* pro jepts , and allowed them to. 
.work autonomously.. Each person* dev,eloped lessons indepen- 
dently. One member of 'tHe^group expected to.be appointed 
leader ' eventually . WHerw a -proiect director finally did " 
arrive', several months later,. nis efforts £o-prganize and 
implement a work plan were thwarted by/ihd tyidual members* . 
who 'had vested interests in materials /already developed and 
In directions already taken. Productivity of the group fel 1 
arid rnuch time.vas spent in behind-the-scenes struggles' flor \ 
power . . ' ' 

Case 2B1.2 - Temporary adrninistrato];' 

y A project was tasked with implementing an exper injental , 
cur?4»curium ^requiring a* variety of high-level coghitUy.e and 
af/ective skills. The project was expected to last for three 
years, including training of 'staff , d^velopme^nt of CPE 



materials ahU evaluation of their cost and- ipstruc.tional 
e/fectiven^ss. It was difficult '^to find a suitable indiv^r 
dual to oversee tht curriculum' development since highly 
techni'^cal expertise as well, as ciifriculum development skills 
were needed., ^ The curriculum developer joined -the project 
six inonths 'after its beginnihg. -Meanv/hile » a temporary '.^ 
leader JiraS* in charge. He was faced with two alternatives.' 
A]|||^hough not qualified^ he could mak^»»f'undaFientai .dec isions 
for the project whi6h Wpuld all^ow it- to begin productive 
work from the beginning. AlteVngtiyelyj ^he could postpone 
these Mdecisions until the arrival of the curriculum dev^l- 

^Qper, and allovr the pron-ect's authors to develop lessons in 
Vhat|rver areas-'they felt rr^ight be* useful to^^the project.' lie 

, selected the .Second aligrnative.- , , v . 

The lessop^' which *wet>e, developed in the si:x ^lonth 
period before the otir r ipul^jm^ developer, arrived 'were for* the 
most^p^Yt unus-feable; thejr sirriply did not fit in^o his plan 
for the project. . Ultifrmtely th^ project failed to meet its 
intermediate! and' fin^l* prorfMction deadl ines .\ The inability 
to use , all of the pro ject • S time effectively' was prob^jbly- 
detrimental to^ its successful jcontlusion. Moreover, the 
early loss of tirrje , f orced a hasty and >sli^sbod manner of 
lesson .development in' attempts to meet the deadlines. 

2. Diverse qual if ica tion? ; of d irectors , 

Case 2B2.1 Con tent' expert studied CDE ' ^ 

In a'^t leas-t four projects the director was a -content 
e^^x.pert who^ had teaching experience and/or had taken courses 
in instructional design but had no. expe;rien<ie v^lth CBE\ 
R^hfer' tKan begia staffing and rely on 'experience with oth*^r 
media for setting proj-ect policies and goals, the new 
directpr 3pent one or .more years , learning the TUTOR language 
Sufficiently well to prod'ude and te^t several alternative 
tylpes of instructional .approaches that might makeTull use' 
of t^ie » m^d ium . * ^ 

Therefore before he*^ began full' staffing he had a good 
understanding of the al tern^tives .in Reject development and 
was able foi^mulate clear go'als. The pV-ojiJcts*.. ultimately 
produfed Idrge amounts of good' material on-^a tight schedule'. 
. Thi? 'was so, ev.en though- 'pr9b]jBms sometimes arose in the ^ 
selection of an experienced^ programming staff whose views 
fit in with those of the-' director . . * ' ^ ' ' 



Case ?B?.2 - Expertise in* instructional <^1evelopmer/t 

The- individual who was^ chosen t&^r^ad one project had ' 
experience developing instructional materials for the - 
raudience. and institution which was implementing CBF.' The ' 
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Staff he chose had varied backgrounds. Some were experi- 
enced teachers apd others weFe subject matter * experts . ^^^ly 
one other person had experience developing instructional^- • 
materials. The lessons product under his direbtion were . 
for the mpst part effective* and useable. The experience of 
this director in the production of instructional materials 
seemed to make him aware of the need to m^ke maximum use of . 
all available time. When delivery of term'inals for his site 
was delayed, tve'used the time to teach the basics of ^ , ' 

instructional design to his staff and planned for'sharing 
terminals with a nearby ^ite.'' 

Case 2B2.3 Non-adaptive applical^ion of expertise 

Th^ pro ject . leader was experienced in innovative cur- 
riculum development, particularly for special subgroups of 
students. He decided'to transfer the old , materials to CD^, 
using^his same less9ns and 'instructional strategies.. He did 
not take ijnto account the differences between CPE and cTass- 
room interac:t tons . ^ ' Many of the lesions wef^e not suitable ~ 
and students rebelled at doing" them. Many' lessons had to b,e 
scrapped or essentially rewritten. Considerable-'time was ''^^ 
wasted and tffe project fell behind its goals. 

Case 2B2.M - Superb- programmer " • • * 

An ad<g)Lnistrator appointed a pro^j'ect h^ad who was " • 
experienced in *a subject area related to that in'tended for 
'CBE, but had no te.aching experience with the target 
population nor with -the exact . sub ject matter.. .He was » 
apparently a hotshot in programming. The resu-1^ was that he 
directed the efforts of the project to develbping an exten- ^ ^ 
slve comput:er-managed instruction capabili'ty. ^Insufficient 
time, was allocated to writing instructional ma^Eer.lal. There 
was tiot enough- time to produce ^3 m^any lessons^'as wefe * - . / 

needed, and very little time, was aJlpwed for student trials. 
Many of the lessons that were produced .turned out to. be • , * \ 

jjnsuitll^le for the intended studentys. ' . 

*■ , . 

Case 2B^.5 -'*Ex,^ertise ' in administration ' . ^^I^^l^ 

In several Instances, the pro ject ..Supervisors were ' ,r 
administrators who had neither ' exper ience teachingthe ^ 
target population nor subject matter expertise^. They oiade 
ttie decision to discard certain lessons, baSed on^their 
Reactions that* the lessons were too. "bo^ring^ or "too easy." ' ** 

'But tlie authors felt that the presentatron 'Was at an appro-t ^ 
#priate difficulty level for the students. They convirjced ' 
the dire,ctors to allow >the lessons to run and to let the 
data be u^sed. as a basis for deciaion. Data showed that " ' . 

{ ' ' ' / 

,^ . , ' ' ^ • 
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student% liked the lessons and also had favorable , 
.atti*tude.. The lessons werte refeaine<i and^used\ - 

3. Rart-ti/ne leadersx of large projects ' ^ r 

Case 2B3-1 -Too many duties • • ' ^ 

An individual ^was hired to be general supervisor of 
several QBE projects and to be directly/in oHafge of one 
of.them^^n additioi\, he 'brought with him f/om his previous 
position some continuing pro^ject%. He decided that' he- neede^i 
i time\<to learn to wri-te les'sons to 'get a better "feeMftg f^^ 
%hat was involved. However he could, not find time ^d^^o'ao,^ 
Mi5r ^coiild ^he even find time to •observej>sta'dents working at 
thp' terminal^. . In many cases he made ill-^ised^ decisions 
an^ judgments about what k'indi /of TesSbns would be success- 
ful. Staff members becaThe increasingly |;'eluctant to a^k for 
his. opinion on such things and worked independent^* Even- 
tually he'^ had almost vno ,time to devote to the CBE project/' 
The- staff splintered ^into factions andtmorale was low. 

Case 2B3.2 - Coordinator, responsibility withpOt 'aijfhority 

In four d;ifferent ppMects, the leadi^r 'of the currir 
culum gpoup was also responsibly for many other acti(^iti%s 
and had no time to engage in day-to-day oper^ion of the 
group*. He made pcJlicy deci-sion^ yhich v5?ere' in most cases, 
made solely on the basis of reports rather than personal \ 
observ'ation of the situation or the ma,terials being devel- . 
oped. He appotiited a group coordinator^ to give day-to-day 
supervision.- The coordinator delegated assignments but was 
given no authority tp make and^ enforce decisions. The 
leader -^ave no general guidelines and left planning to the 
coordinator. The leader encouraged group members to report 
problems directly to him rather than wqjrk though the copr- 
dinator. Decisions resulting from, such conferences were * 
often revealed -to the .group ^before the coordinatpr learned^ 
of them or- had a chancfe to gi\fe counter arguments. ,The 
.coordinator eventually became ;Wttle more 4^han an assistant 
to the project leader. Daily operation of 1:he group became 
f ractton^^ted sinc*e the group members would only follow 
directioi^s^f the overall, leader who was r^rel^^ present or 
available. A good deal of groap effort was $p^nt iny 
"political" irifighting. There was no overall structxTre or 
framework for^^^on development . The totality o'f lessons 
was not a coordinated whole, but fragmented pieces. 

In some cases .the director Received conflicting 
Tnfof^matlon. Since he did not have time to verify stories) 
he resolved the conflicts by counting- the number of people 
telling the same st#fey. He then made decisions' about^ 



project poiicie.s on tYfr^ basis. Sometime;^' this resulted in 
the over-ruling of, unpoj^^lar suggestiorts that had b.een made 
by'an experienced instructional design^who was noTn'ina'l ly^ 
responsible Tor daj^to-day operation of the group. The 
group went leaderTess, m'or'ale fell,' and productivity was 
almost nil . ' ' " • " , ^ , 

Epilo.g . The entire group was reorganized under a new > 
director. Although the^ new director^ could only devote a • 
fraction of his. time to' administering the" pro ject , *memtier s 
of the groups tia'd gained enough experience so that' they were 
able to work indepen'den tly . ..Even so, deadlines were so ftear 
by the t'im"^ the -groups, Became productive that^only minor ^ 
parts of the original production, goals W^re reached and 
overall quality of tjhe,. mater rals was mejdiocre • at Hest". 

• . • 

^ ■ , C. .iptaff Selection, • 

The effectiveness of each project, was ^ometijnes impeded 
and sometimes enhanced Vby, the way ?n which staff, were 
selected an-d. trained.'/* ' ' V 

■Jn order to- selecfa pro ject /s£gf f , the project leader , 
needed to specify the qualificationJB he was looking. for. 
Early in the'- development of PLATO , pro^^s^^ it was known' that 
sQccessful'^riresson authors were experienced* in more than one 
area, .such as lesson' design , sjjbjeqt^matter., apd 
programming.'^ However 'tjiere were very few people Ground with 
all of these qualifications'. In acJdition trtere were no 
f orrpal training progr;ams '^in the early years , so iird.ividual s 
learned on ^^eir -.own or project directors developed .th^ir 
own trainiq^ in whatever way*they could'. Th6. first formal 

:authqr training* program underwent severaT*"iterations .before 
i,t* was ev^^luated ^s successful thfree years after PLATO IV 

>was implemented •(Fr^'ancis , 1976). ' * 

Director's were no^t -always free^tTo choose the entire 
staff. Sometimes ind.i yid.uals were carried over from a 
previous project. Sometimes hew staff'wer'e Jni^red simply 
because they were available and' Interns ted &t a prfopitious 
time'. Some staff were students who wanted to learn how* to 
author -"PLATO lesions. Gthers were^ individuals with exper- , 
ti-se in perhaps just one ^^area like . programming or teaching 
or instructional design. Still others were instructors- who 
were pleased from all or part;^ of their teaching duties in 
order to 'develop PLATO lessons. It seems that both teachings 
{or curriculum^ design) experience and subject matter exper- 
tise were necessary but not sufficierft qualifications for 
good authoring. Instructors who had only lectured and had 
not interacted with studentaj^on* an individual basis were. not 
adequately sensitive to the sti|den,t's needs. They tended to 
equate teaching with presenting and in.corpor ated very. little 



♦interaction in their lessohs. Skrlled programmers who had 
no teaching experience produced, lesions which were ineffec- 
tive. " Retraining carry-ove-r staff vas-relati vely un^uo- 
cessiful. ' A director who-declded to retrain underqualif ied^ 
carry-over stkff later expres^sed some lyiisgivings . Such' 
staff required greater than average consulting assistance 
and jnany Of the lessons they produced werer poor, or Unu3e- 
able^ In another , incident auth'^rs who had previous CBB 
experience we're retrarined to use; the TUTOR language. Many 
of thefse afithors were unwilling to Learn about the' new apd, 
more, so,phistipgted ^a^pects of , PLATO or to apply that know- 
ledge .to^ producing PLATO lessons. Instead they wrote mate- 
rials patterned after the inefficient lesson formats which . 
they had been .required to *use pi;eviou5jy. * ^ ' 

When instructors were given- released time to write 
lessons, they needed at 'least half-time release to be 
effective. Per^sons with very limited amounts pf released, 
time spent most of ttt'e.ir time in "warm-up" or overhead \ 
activities and contributed little to a project. Even 
half-time release wa^ not always effective. The intent' to 
provide half-time for CBE was ti^nest, but it could not 
always be^ honored. Teaching duties diffeT from CBE 
responsibilites in the, immed iacy of the needs, deadlines, 
and payoffs. When the total work load became^ excessive, 
teaching duties such as meeting a class, took precedence 
over les?on writirjg. " Consequently teaching sometin^es topk 
more than the 50% time allottefJv and CBE lesson writing* 
suffered accordingly. On the other hand, full-time release' 
was not always successful because the instructor^ he/iame 
isolated from the other instructors and from the problems of 
tfie .students . , ^ * ^ ' 

» A decision specific to the '^mi 1 i tary environment was 
whether or not to employ civilian |staff. The decision, to 
choose both military and civilian "uthors worked out satis- 
factorily, at 3 of the 4 'sites observed. 'At the fourth sit^, 
pay differentials for equivalent w6rk^ caused an exodus of 
military authors'. ' V ' 

1 . Qualifications 

Case .2C.1 . 1 — High turn-over jsubject .matter experts * 

Th^ director .at ond sit? w*as not free to choose his own 
per'spnnel. to author CBE' lessons. Mo3t of\his staff were 
subject matter experts (SME)on one-year assignments to a 
training organizfftjlon . Tkle SMEs wefe trained in TUTOR, and 
programmer's were 'Hired to help them^ develop lessons. The 
programmer's we're effective in helping them^ but the ratio of 
T programmer to 4 authors was inadequate to meet the 
authors' needs. The lessons produced were essentially like 
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textbooks.. One frame ^a^^ter another was an^extewAive di,spla^ 
of text. Very little interaction was incorpp^rat^ ipr'the- 
l essons , - , , 

Case 2C1.2 Teaching. experience - . # # , » - 

* , Iij anoth^r^project personnel had already developed use- 
ful .lessons . A new d ir ector *ari||ved with Some of hi^s- fornier 
staff memb^s-. They/had teaching, but not CR^ ^xp'e rience . ' 
They tried to make CBE fi.t old molds, ref uilipii^to learn from 
experience 6f exiS'ting staff Thi s . resulted . in bad rel*g- 
. tionships between the c^ld and new staffs and ultimately, in 
inefficient program implementation. 

Case 2C1. 3 - University , students . enroJ.led^ in - CBE-course 

Students enrolled "in a university cours.e for the ' • 

^specific purpose of learning Fipw.to write lessons on the 
*PLATO system. Since the • prof essor^^ s d^epartment 'had no funds 
for developing PLATO lessons, yfre decided - to-'have thfe 
students write lessons that ctuld be uSed for a beginning 
course in his subject. - ^ " * ,^ , 

Ihe result was an exceedingly ctlv^erse set of lessOns^b^* 
varied quality, depending on the abilities 'and motivations 
,of*the student author. Some lessbns .contained g/'c/ss errors 
in* content. Staff niembers w^e able to'get lessons revisecT, 
Xor content accuracy, but it was'-^often difficult if not 
*irmpr?5ssible io get , the student , author, to make other kinds of^ 
revisions, such as spelling, or more reasonable answer- 
judging* The first time the lessons were used for a regular 
class, the CBE stu^lents indicated ttiat* they enjoyed Using* 
the TLATO system. However, they did not do as well as the 
^s.the non-CBE studentsfT>n exams.* The instructor ascribed 
this performance difference in part, to lesson ' inad^quat:ies ' 
ana error:#* </ > 

Epilog . The instructor subsequently decided to revise the 
•lessons himself, at a considerable^investment of time. 
Classes that used the revised versions then performed afe ' 
weir as non-CBE students. 

Case 20.4 - UniversTty students extracurri-cular* activity 

• Department fundjS were not available for developing CBE, 
les^cuis.^.^-J'he. decision' was made to have stud'ent programmers 
write.J^sS'ons . ' ^The students seemed to learn the material 
they programmed themselves, but the lesSons could not ^e / 
used to teach other students. An inst rue tor f i nally ended' 
up revising some of the more promising lessons and* supple- 
menting "them with lessons written o,n his own time. ^ ~^ 

y ■ - 
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2\ Special consi'dferqtions- in njilitary environment's 
Case 2C2.1 - Both military and civilian authors unsuc- 

* . At a military base, the administrators were uncertain 
whether civil , serv^ce'^ St-aff or tnil itary staff wOuld make 
better authors. They, decided to make a small research ^tudy 
of the is'Sue and to select half th^staff fbr the project 
from each an^ to study- t^he differerWes. later*, "The civilian^, 
had comparatively precise job'de^cr iptions and a 'higher pay 
scale than their;' military colleague^/ ilfcen the system was 
"d5)wn", traditional course autjjors complained to* the ✓ 
director that the CBE authors were just ."sitting around'.". So 
the**director imposed demeaning clerical ..and janitopial work 
on the CBE staff group during down tim^. The civilians v^^re 

.able to point to their job descriptions and* thus escape — 
.i:doubling the "nasty" work for the military personnel, this, 
ecmbihed with- their ^ lower pay,\created a substantial morale 
pr^lem tor the military authors: Many left or trie'd to 
leave before the project's end--thereby causing substantiaT 

.problems, and delays while new ;staff werje identified and 
trained. '•v ^ . . - 

Case 2C2.2 - Both military and civilian authors, successful * 

At two other military ^MtesV civil service and military^ 
personnel ^workecL together without major problems. There 
were apparently small- differences in pay and virtiially no 
differences^ in duties or responsibilities! ' 

Case 2C2.3 - Al^military authoris, personnel changes 

One military" project was acquainted with the situation 
- des'cribed in Cas« 2C2'. 1.- So they decided to' have an all 
military, staff., 'Host of the selected staff, wer'g officers, 
gnd thus they did not ' encounter the problem *of ,demeaning 
♦janitorial duties being foisted oh them, as at the other ■ 
.site-. Obviously nd* fhil itary-vs. -c ivil ian .problems arose.. - 
However, this project had to deal with the severe" .problem's 
which any group whic|h includes military perspnnjel must be 
prepaTed fo-^accept. That is, promotions and reass'^^nmenf of 
.personnel forced reorganizations of 'the CBE project, with 

'attendant !^ss ofsef f i'ciency . I . ' . 

Case" 2C2A - All^ military authors, pl-obl*Mj»due to rank' 

At- a .military si^, the external .supp^tive^ irtstitiitlon 
encour&nged the director to include instructional cfesigner^ 
an?l evaluators as -part of .the lesson development group ' 



Consequently,' an experienced lesson deisigner was assigned to 
assist the CBE authors. He held a lower military rank than 
tlie authors. Even though 'he was very « comp.etent , and the 
authors would have* written better "lessons if they had 
followed his advice, they largely ignored him because of his 
lower rank. A different instance of the* effect of military, 
rank is given -in Case 3A3.1 b|low.. ^ 

31 Carry-over staff from anVeced^nt projects 

Case 2C3.1 - Retrainings existing sta^f to be authors 

One project -was set' up enuring a period* of tight organi- 

/zational fundin-g and a s'^hortage of local manpower. There 
were no funds tt» hire new staff' and the existing pool of 
Unassiglfed personnel from which to staff the project did not 
contain qualified individuals. The proj^ect director was 
advised he had two choices: employ the under-qualified . 
staff and use them as best he could, or' employ no ^one for* 
those positions for the term of the project. That is, the* 
positions could not be held open so that. better qualified 
'^taff might Jater'be ad'ded. . . ^ 

.the director decided tht slow producers would be pref- 
^erable to understaff ing the project and 'hence filTed all 
his positions. 

.The shortcomings of ' the stdff farced the director to 
revise^is management of bhe project. He devised *a team- 

. oriented approach so that the weaker aqtl^ors could serve 
mainly as .subject matted experts^ One person was removed 
#^tire>y ff'om programming/subject-matter duties, arrd glv%n 

^clerical/editorial responsibilities until his position could 

r^^e terminat^ed. |^ 

Epilog . At t later. time, the director and ^is assistant 
expressed s^me misgivings about the decision. The Und^er- ^ 
qualified*^ staff consumed, an excessive amount of the time of 
on-site and off-site consultants. Their work was sometimes 
unacceptable and required dom'plete reworking by more- 
qualified staff. But the* fact that some of the staff were 
well-known in the institution probably enhanced the accep- 
tance/of the ma(^terials. 

-•Case 2C3.2 - Failure to adapt to new features 

Some members of the staff at one site had useS another 
CBE system.^ They often tried to force their PLATO/lessons 
into- old molds and ,were reluctant to use mpre sophistlca^ted 
and powerful techniques on a new system.^ For exarhple, they 
did not use the data-keeping featuKfts of the PLATO system 
but rather devised cumbersome", less informative,* but • 
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familiar routiiies similar to those 'which been used on 
their previous system. Some staff had Werf editors of 
programmed text. Initially they attgittfJ^ed to use CBE as a 
■ prog;'ammed text ignoring the^ possib^Qity of using • feedback . 
The ^lesson^ were boring and ineffective. 

4.^Staff from within, released full-time*^ 

Case 2C4.1 - Isolation from other instructors 

Instructors 9t a college were given full-time for 
production of materials and ,compl^te freedom in set>4ng ' 
, objective's , although tjhey had rto prior experience in either 
instructional design or CBE. Although they be'canie^ profi- 
cient with the CBE language, tjieir total lesson .productivity 
was not impressive. Moreover, they were perceived by other 
members af their departments as "outsiders^' and had diffi- * 
culty in^eheduling trial'use of the^ir materials by stu- 
deiits. fn one instance, a person produced^ no material at 
all during' a year of released time and Jeft the school f3r 
another position near the^nd of the' period. - 

Case.2C4.2 -''Isolation from students 

Reg'Ular instructors were released to be CBE authors but 
they Ij^d .almost no experience in any aspect of QBE. They^ 
tended to' produce material without attefmpting to test it 
. with students or relate it to student needs. The rate of 
production was also rather low because deadlines were in the 
distant future . 

, Case 2C4.3 - Milita^y getting, .isolation ^ 

A numbeV of instructors at a- site were chosen to become' 
CBE authors. They tended, to be the younger and better edu- 
cated of * the' group. Those who were not pickted thought that 
the CBE people were -somewhat of aft "elite" group and. had the 
' easier and rrrore interesting job. The CBE staff no. longer 
did any 'teaching an*d their lesson writing bepame isolated, 
from the .course. The lessbns they produced were of- variable 
quality. This situation, Qoupled' with admi^nistrative pro-. 
<lems, th.e. small flow of students, and the fact that there 
were no real problems teaching the course in the traditional 
way, resulted in -little use' of * the'desso'ns . 

At another site wit-h a, nearly identjical situation, one 
of tj^e authors, making an exit interview,* stated .he felt , 
that* one author working full-tigie would produce less than 
two authers working half-time, - that the latter . would 
better be able to relate the lessons to the student audience 
and to smoothly* impleme^it 'the lessons. - , 




5. Staff from' within, released part-time " ^ 
Case 2C5.1 No CBE ^^perience • , ' t 

in 2 college level courses , instructors wi<h nd prl^Or- 
insb*tT<i^bnal desigi^ or CBE experience produced little mate- 
rial during an initial -year of 507;-releas^d tirne. ' But they 
were abte to test those matjerials within their own eour's'es 
and tleVe^op effective instructional .approaches which led to 
p^ro^ductibn of substantial amounts of . material in later 
years. 

Case^2Q5,2 - CBE experience ^ , . • 

J " * 

in three c6llege level courses j-instructor-s who had 
spent two- or^more y'eafs .developing CBE materials on their 
own -time were given halfV'time release td consolidate and 
supplemerit materials already produced. Productivity was 
quite high in two cases dnd in all cases/ the^eleased .time 
resulted in substantial 'additions to the amount of material 
available to the ^departrpents involved. 

Case 2C5.3 - Less than half-time release ^ • 

Regular instructors were released M/M tiofie to, develop 
instructional materials for CBE. That vfas not enough time 
for .them to write lessons. The bes-t they coOld' do was 
review ^lessons that ^others had ^written ar)d somj?times they 
could do little of that> Almost nothing useful was accom- 
plished/ V . ' ^ . 
V In 0T>6 instance a person with 1/6 released tlm^e 
reported that he felt that he had made no contribution at 
all to the project. This view;was shared by -other members 
of the project who felt .that the efforts in attempting to 
train him had^ wasted linjited resources. ^ Attempts, to avoid 
this problem resulted in a ran^e of released-times tried by 
a number of projects. 



n^^ 



t ' D. Planrfftig the Project 

In order to ^schedule CBE lesson praduction, project 
directors needed infqrmation on wTnich to base plan%. The 
only data tpat were initially available were based on a 
'special group of higjjly-exper ienced workers. Th-feir data 
were citi&d in respon^s^^^^flu-i^h^stions from director3 of^jiew 
proje^cts, but the fact that LKe. data were^for experienced 
workeVs was often not communiciated effectively. Even aftir 
more complete, data became avai lable, -figures' of 30 hours of 
production time for one ' hoTlT^f. student contact time (valid 
for one set of very prbficient autho'fs) were cited rather 
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trtan figures of '^several hundred k^tp^s per contact hour 
(valid far new authors during^ their fii^'st year). Conse- 
quently^ planning f or' product ion , when done at all, was 
oftefl predicatec^ -on production rates that were impossible 
and unrealistic for the untrained workers available to new . 
projects. ^ , ' ^ . . 

Some directors made 'overam^bitious plans, which l^d to 
failure to meet/deadl irtes . In cases where no contingencies 
had been P^laj;ined and deadline-^ could not be met, the deci- 
sions of 'project directors' variecl from just doing the best 
they could under, th6 circumstances \o giving up lesson, 
writing entirely. and writing routers for .managing 'l essons 
which had been written, by ot^hers. . / 

Administrative organizations of the projects also ' 
yaried considerably. Some projects, particularly rn the- 
military, brought together a nujnber of indivi^u^s to pro- 
duce a fixed numi)er of lessons in a given lengtji of. flme. 
Directors tended* to underestimate the time th^Fvould be 
hfeeded and soiffe ^Iso failed td arrange for, contingencies in ^ 
casp they would-be unable to. meet their goals.' The sue-' 
cessful projects were the ones which were planned so that. \ 
lessons* could be effectively wsed even if goals were'not . 
completely met.^ - ^* \ ^ 

Some, projects evolved- gradually, wJth a single indivi-- 
dual^^first gaining experience writing indlvidual^lessons 
'iand later jon as^^mbling more staff members and /creating a 
curriculum.' Many of Jthese were , highly sUccessf-tiVpro jec*s.. 

Another administrative question that had to be, 
addressed by a CBE project directoip was how much planning to 
do ahead, an^ hd^w mucli to leave open* and subject to the 
discretion of the author. This was' particularly relevant 
for projects tbat used the PLATO system, where it was '.a 
simple matter to "compose" and/or revise lessons , at the 
terminal. The system does not. cohstr^in the' lesson^ , 
developer to make, long ran-ge cgmmitments to, a f ije^^ plan 
which will be difficult to chafige at a later time. EarTV* 
pr^>jects varied from detailed planning to some general 
planning with j-ust a few details prespecif ied , to almost 
complete improvisation. The two extremes were apparently 
the least productive iri the long-run. , . 

The selection of appropriate media was an important, 
consideration irt'project development. CDR is not* always the 
most suitable .medium for all instruction.. When project- 
directors Tjsect PLATO instruction, exclusively- without consid- 
ering the appropriateness of such^'use, the CRE lessons were 
frequently inef f ective^^,.^r in^, and' not cost effective^ 
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1v ProcLuction goals ^ , .-^ ^ \ 

^ Case ?D1.1 --Working from the:'end..^f coursie. to 'beginning 

^ At one '^mllitary^ si te ^ plans were ''drawn up even befnr?* 
training was bef^^Tfin the programming .language, Sincje ^-a 
standard^ course of irist):uc\*on was already ^in* use , t>ie task ^ 

^f*the prx)ject was to develop parallel^CgE ^it/stru'cYioti^ The 
decision vas made to' wi^ite the ,last^ lesson/ of the\ course 
first and then cQ<i£inite to. write ,iri"^^the reverse order in ♦ 

'.wtaich*the lessonS. wdyld be usetj^^ Several 3dvantag«s 

• resulted. The .student did* riot h^ave^^to wa*it for -a lesson . 
untif .the whole course was ready^' He s^impVy worked in' the 
standar<j mode until he re5c+ied the point where, the CEE 
les3ons werec ready, and then continued with tl^em until the 
e^nd.' There .were other- advantages to this plan. The first 

- 'CBt' lesstSn the student encoynters 'sets his at^titude toward 

• thp* nediuni. Since the aijth'or ' s ""f ir st-^J^sor^s are apt to ^be 
his -worst, it^s b.est if if they are not the first CBE 
lessons .the student ,en^coun£ers. In the plan under discus- 

• gion ,^ the sti/dent dfid not*", encounter the 90thor*s first "les- 

• sons until the end, by wh.ich 'time he was hopefully tolerant 
5)f its shortcomings and more adapted to the medium. 

^, Finally, evalua^t ion' w*as possible even though not all of the 
; lessons originally planned were ct)mpleted. 

Case 2D1 .^2 FTix^^d number of hours , no ' contingenc ies 

Some projects were given some latitude in setting their 

' own goals. Thus, ^theywere, free to set goals in terms' of 
t(^pic9 to. he covered or in terms of student contact iiours 
they would>'provide . - , ' ^ / - 

In 'two* instances, inexperienced project directors set 

.'^oais far Beyond the resouroCes . of their groups and* failed to 
make any attempt to provide alternative goals should the 
planned ones pr.ove ^.mpossible. Despite continued slippage . 
,0^ deadlines, the planned goals were maintain^' until 'final 

^ deadlines were so close that it was obvious to all that 
there was a real cHanee for total failure of the proje.cts. 
As a last-minute alternative, all attempts at lesson prod- 
uction were ^bandoned and lesso'hs written by , other groups* 

X were adopted (despite: the fact that many of these materials' 
had been regarded with contempt a few montj^s earlier). 
Ultim^ately, the' adopted , materials were arranged in w§dl- 

' .docum'ented packages .that were accesse0/^y " speciallj^esignecl 
routers. While the^final "product'^-^ttas a reasonable outcome 
of the project, it*would probabty hav^ been of far higher 
quality had the or'iginaJL goal been ajltered much -earlier . . 

4 . ' \ 



Case 2D1.3 -'Inappropriate basis for'prcicluction <^cl)<?clul^ 
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The decisicxn- was madq to develop lessqns which would 
supply a ^pecifie(^ number ^<5f student contact hours. Pilans'' 
were erroneously basecj^ on planning guidelines which* applied 
to experienced rather than the- inexperienced authors; ^ ' ' 
employed on this project. 4lS a result, the director^ ^grosjsly 
underestimated production time requirements. He further 
failed ^0 provide oaltern'ative pilans should the original 
goals prove impossible for any reason. When it became 
^obvious* that the orsL^inal goals were njrrattainabl'e , bh^l^/w^re 
" substarttially^ scaled down (and. contingency plans' were^rrad 
for future problems). This alteration of- plans underpres- 
sure of potential, f ailure (rather than as part of a planne 
sequence of alternative goals) wa3' demoralizing' t# all par 
ticipants of the project and tended to >lead project direc- 
tors to engage in "cover-up" and -blSwie-Shif ting tactics 
rather than in activities directed toward improving instruct 
tional quality. ' ^ ' * \ 

Case 2DU^ - A complete set of lessons / ^ 

^ Flaws for one project ^nta,iled ,a new curriculum to be 

devel€>ped on a new medium, CBE. All the lessons in the 
course" had to be, completed in order to judge its effec- 
tiveness; This virtually eliminated the possibility of ' a. . 
^partial"- success, since unle*ss all the lessons we're com- 
pletjed there could be no projfect evaluation^ When, it .became 

, apparent that all of the lessons would not be completed by 
the deadline, the entire project was scrapped. Officially 
for other reasons . However, the lmp5ending failure t6 
complete the lessons wa*s a major contributing factor. 

Case 201.5 - Incorporate lessonis as they become available 

« ' * 

In^at least three pro jects , .the production goals wef e ' 
s^.ecified in terms of lesspns covering particular subj,ect 
matter topics.-. At first ttfe project director was the '6nly 
author. As soon as I'e'ssonfl were "finished" they were Incor- 
porated into courses in appropriate Tplaces. Eventually, 
t^ie director added some staff members. Although completion, 
deadlines were * not .strictly pre-speci/ie^ , ^ production * 
progressed wel*l . This was possible because the 'directors 
themselves, were compejtent, highly motivated authors who- 
stayed in constant contact with their staff r Ttjey set 
realistic goal? but in ^an informal,* flexible setting. 
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Case'2D2.1 - Isolation of instrulstor from CBE ' 

Curraculum materials were writterf at a development cen-' 
t'er for /impTlement'ation elsewhere. The pro ject director had 
the alternatives of making the program comp-letely self-, 
contained or requiring instructor intervention. He chose 
•the Tormer. He therefore decided that the entire curriculum 
hatf to be mapped out before any Student triafls were- . 
^attempted. This" included writing every objective and 
'routing cojit inge^nisi^ . Since the full weight of instruction 
fell on the CbI i^gterials; a 'great deal of time had to be • 
spent by the -staff in altering and maintaining the elaborate 
routing and branching atrjifcture^. .They did not have time*" to 
revise the lessons themselyes or to add new'^ones Hjiere 
necessary. Since the lessons had minifnal pretesting with 
stu(^nts, the/y often needed a good !deal of revision^if they 
were to be effective. The instructors felt that since the 
router was'doing all the work, they did not need to get 
involved. When problems arose they did not try to^' find 
solutions themselves, but rather callecj on the CBE' staff. 
^Furthermore,, they felt frustrated because the lessons were 
not meetin'g the needs of the student's, and t^ere^-was nothing 
they could do about it. , . . - * ' ' 

hpilog . The pouter was scrapped and a new one was Written. 
This one made it the teacher ^ s ' duty ' to select ' lessons for 
the* students . The task for the teacher was made as, simple 
as/possible. AlthougX this made some extra work for the 
teachers, they l?ecame\incf\easingly involved in ,GBE as a reg- 
ular part of instruction". Also, as they becam.e more familiar 
with the terminals and felt more at ease with them, they 
began to try to solve some of the problems themselyes. The' 
instructors* calls for help from thf CBE staPT decreased 
noticeably, and ' it ' became possible for the QBE staff t© 
concentrate more on improving the instruct^nal quality of 
the materials. 

B^/Pailure. to define goals 

* « 

Case 2D3.1.- No plear goals • ^ 

At one site, authority for decisions on project' goals ^ 
was divided between two groups.* It took foifr months to get 
approval of an Initial set of plans. Dur*in)g that time thef 
student population changed. Moreover, the plans turped out 
to be impractical due to lack of experience with CBE at the 
time they were, written. New 'plarfs had^ to be . f ormulated"! A 
major obstacle to approval was a conflict in the goal3 of ■ 



the two groups: One was interested in Seeing CBE deliver / 
instructian; the other was ..research oriented. The project 
thus went along without clear objectives. Lack of decisive- 
ness resulted in ^refluent chang^ of> plans. For all practi- 
cal purposes, every time a plan was chanf^ed, the progress in 
the previo.us^ period was cancelled. .The net effect was v«ry- 
limited progres^^^or the project. ^ • r • 

- ' • 

Media selection * , / 

Case 2D4.1 - Failure to be selective 

At several ^sites the project directors assumed that* 
since CBE was ta be used, it had to be done to the exclusion 
of' other media or had' t© be incorporated somehow. whether 
feasible or Inot. At one site, the most effective^ medium for 
a task was Videotape ; -so .tPTfe project director /decided to use 
PLATO by^ having it. control the videotape. In another • 
instance^ an autbor suggested^ tltat a particular' le'sson pe , " 
presented as a handout, but tlie project director decreed 
all lessons must be in CBE. The re^sult was a Usson that 
was boring and costly. * " ' 
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• -Chapter^ 

PKOJhCT ^rVELDPHENT 



• , A. Director's Role 

'As 4^LA'iO pro jects .evolved , some continued* under the 
'leadership of the initiator. Others underWeat One or moi;e* 
changes of directors.- In many larger projects , the organi- 
zation necessitated several leve-ls of authority and in those 
cases administrative structure varied. ,For example, within 
a single institution, CBE les-sons -were' being' written in 
several subject areas. Each group had its own project 
Reader and all such projects were under the supervision of a 
higher level administrator. I|^>sorrfe" cases the top level 
director. was also the director pf one of the ^su^^pro jects; 
in, others he was ' essentially a cooirdinator and administrator 
* 'in management rather^ than in' curriculum development. 

Unfortunately, roles and domains of authority -were not 
always prespe'cif ied in these mylti-level authority groups. 
Leaders at the same or parallel levels of authority could - 
and sometimes did intrude on the projects of oth^r^*, giving' 
conl'licting directlv^e^. When th is < happened , staff were 
con'fused ^nd progress wa5 hampered*^ - \ - 

As noted above , leaders who chose to be in constant 
contact with proje^^t were a positive force^ in successful 
♦development and cbuMffoften pr^^i^Tit problems from], 
developing . • ' . . . • , 

Pr6jeot progress was sometimes" slowed -due -to a change 
of airectors* In at least two cases, this .happened because, 
the new^d ir^stpr . ignored the accumulated knowledge of his* 
predecessoT%and^ started frotn scratch. 

^ A spacial proble'm. in the military situatfLon arose when, 
promotions ' raised a- staff menrfber * to a rank* above that of the 
project director. The new senior officer assumed responsi- 
bility according to military code, and the structure-, roles, 
and responsibilities previously set up*.were left in 
- disarray. ^ ' . - 

1. Keeping in touch - ^ . 

Case 3A1.1 - Positive result of support , . > . " 

^ At one laVg% institution, CBE lessons werf being de.veL- 

oped an several departments. The top, adminlistrator , who had 
overall responsibility for the individual projfect^, chose 
♦tjiff for each'-'pro ject and kept in 'personal contact . wi'th the 
sta£f and projects. This was a major factor in helping hijfi 
develop a viat)Ie »CBF group in at least one department. 
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Case 3A1.2 - Infrusion 



A number. of projects were h^edquartered at a 'Ourriculiam 
development center^ Authors worked in -proximity to those- 
f-rom^other groups &s' wel 1^ a^. their own/ A' high-f anking ^ 
member of thje center staff took ar well-intentioned intelfest ^ 
in project's for which he had no responsit)ility and made-- 
suggestions which were interpreted as (5rder^ and* which 
sometimes coaflict«d with those of the real. leader. The ) 
result was* cqnf usi'on ^ on th^ part of project members, divi-* 
sive power struggles 'in which sia^ff members pitted the' word 
of 9ne leader against ihat of the other, and lower produc- 
tivity. ' ' • * 

,2. Effect of changing directors 

Case 3A2.? - Reinventing the wheely 

'In tv/o cases, aft*er'the project had been,ander way for* 
:^ear or more, the director left.^nd a new one was ^ 
appointed. , When th^'new director arrived. He began the 
project anew, as if nothing had been done' previously. He 
was<not intey-ested in learning from or building dn previous 
experi'en^ce. Often these new directors brought in their - * 
^>wn staf^ at better* pay than ~the formed staff even though 
the new staf-f was inexperienced in the skills needed' for CBE 
design and required training by the old staff. Morale and 
productivity of the old staff fell, many mistakes were 
repeated, and time was wasted. ^ ^ 

3. Military role conflict ' ^ . 

C^ase 3A3.1 - Military role conflict 

The director of a QBE praject * at a military site-was 
cJiosen for his background and ^ experience in admirajjstra tion . 
He was hired early in the pro jept. before other^staff were 
present. H6 designed a structure, established roles, 'and 
began the project. Several months later, one of the staff 
members, an ^uthor, ,was promoted (on the basis of previous 
service), to a higher ^ank than that of the project director. 
As the ranking officer, milJLtary code held him responsible^ 
for the operation. Role disljinctiwris .became confused as the 
Tines "Of authority w§^re rearranged. * Per^sonal i ty d if f erencee 
between the two leacfers^added to tl3A)stre5S. > The original 
-leader w^b- greatly liked and j-espect^d^ by th£ staff while 
the, new, ranking leader .was disliked. The continuing pre- 
sence of the former director on^ the' project alsd^ added to 
the conftisioii of soine staff members. • ' . 
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1 Case 3A3.2 - Non-supportive aid • 

One consultant who v/as a good prt)grammer .and whp had 
done most of his 'consulting with persons interest^^h-iw com- 
.'puters rather than instructional d'esi-gn was highly critical 
of a.nyone wtio did not readily grasp the CBP- l^anguage. ' One 
trainee who ultimately turned out to be a very effective 
instructional designer was ready to quit after. being tola ' 

' that he would ^never be a good authpr" by this consultant., 
A more supportive, consul tant who was more oraente;^^ tow#rd 
Instructional design was able to -salvage the sittuation at 

. tiXie last minute. " * . . ^ 

. ; B. Staff 'Considerations' 

The software; staff for PLATO IV declined to make 
'available a printed mamual, about the TUTOR language. Such a 
document had been prepared for PLATO* III. Hdwever, since 
th-e 4ystem'was continually evolving and changing, the manual 
soon becajne incomplete and parts of it became obsolete. A 
hard^copy manual for P^ATO IV would have had the same " ^ 
shortcomings^'^ It was precisely these" two fa<:tors, the 
evolutionary nature oY the "programming language and tjie lack 
of a printed ^anual, that made learning TUTOR particularly 
problematicar for |tew authors. Consequently' a jstfong sup- 
portive environmen-t was important and contributed h.^vily to 
the progress of successful 'a^utl^ors.' Many of the best - . 

^ authors ^ere trtrined in an apprent ice?hip-type relationship* 
•with experienced- aut^ors. Other projects found that author 
productivity was impr&ved when^the services of an on-site* 
consultant we?e available^. In several incidents, where 
w^ul'd-be authors made slow or little progress, papt of ^he 
difficulty could be traced to the lack of psychpOpgical 
support. .They were expected to learn authoring by them- ,^ 
selves without the^regular guidance or consulting services 
of an experienced individuals" ' \ 

Some pro^jects* vece organized to include lesson 
designer-programmer teams. When the , progrannners were 
•students, hired on an hourly basis, pen-sonneT turno>ier was 
high and progress was impeded. A more successful arrange- 
ment resulted when the programmer was treated Like^ a profes- 
sional: held a regular appointment', (preferably at least 

^ 2/3 time) and regarded the work a*!5 a job* with top priority 
oyer other activities.- Ope other factox that fostered-' 
smooth relationships was the. decision ttiat the lesson- 
' designer think the lesson through caref ully , gjnd perhaps ^ 
design a few displays before turning it over ^ to the pro- 

, grammerl ' This system minimized the number of revisions that 
were needed later. The programmer no longer felt that all 

^f his.prtfviogs work was wasted because tots^l rewritejS were 

^^arely necessary.* ^ 

3^ . 
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1. The need for a supportive environment 

Case 3B1.1 - AppreYiticQ^hip relationship ^ / - . , 

In two subject; areas, ^project directohs initially* 
developed . their own CBE lessons. After these les^sons were 
. in use iyf students, the directors ''hired additional personnel 

• to wri1;e lessons, ' The^ did not provide a formal training ' 
course. Instead, the new authors reviewed many of the ' ^ 
existing lessons to get some concept of the scope of the 
rtssons and the capability of the CBE system.; Then they < 
learned- the TUTOR language on their own. -^/hen they began ^ , 
writing their les'sons, they worke,d 'in an/ §ippren^iceslFiip " 

' ^relationship wi'tfi the director and other experienced % , 
'authors. The new^staff member could relv on bis mentor as . ' 

* someone who realLy understoo^l the complexities of CBE 
authoring^ and who was supportive qS^'^^M^Aher tri'al-and-error 
effort's as well as ^initial successes and Yailures. This 
kind of relationship was a positive factor for the author's 
morale and productivity. , - * 

Case 3B1.2 - Senior progrsftmer as part-time consultant 

In several groups a senior .programmer was available for 
quick personal consulU-ng for staff who w^re learning TUTOR. ^ 
This ^enabled them to make reasonable progress while 
. learning. It also pr^vent^ed needless frustration. In &ddi- - 
tiqr^ the experienced person could guide the new author by - 
i^icating. which par%s of the language to learn- immediately 
^ana whfch to leave for a future t*J.me. , ' . ^ ^ ^ 

Cas.e 3B1.3 - Different roles of con'sul tants"^ ^ ' ^ 

\ ^ One project manager decided that his group- needed the 
'services of an on-site programming consultant so he hired 
•one. The authors were glad to have Xhis help and lesson 
development progressed' well . HovJever, the manager- expected ^ 
the consultant to also serve asj^a monitor to -improve lesson • 
quality . The mana&er was disappointed that the consultant ^-^^ 
'did not .assume this additional responsibility. 

At another time, a member of the (iERL staff provrded . 
U^t-time consulting services. He worked at a personal-, - 
r^ther^ than at a group l^vel. This* was necessary because 
the freq.uent change in leadership meant that the only long- 
term ^months)' relationships that could be .formed were with ' 
the^ authors. Eventually the Consul tant • s Opinions and " . . 
advice were used* ^s the basis for ^he struggles for leader- " 
ship. When a firfal managerrjent structuf^ wafe establi-shed , 
. the consultant was exp«lled because of the Mrception that 
his role and inf luence-^w^Pe disruptive. ^ ^ 
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Ca^es3B1.4 - Ho programmer consultant 

. A site director obtainie^^ four PLATO te^r^^a^ls^^ because 
L he wanted to provide access for both staff membeV.sv and • 

s'tudent^s. He verbally encouraged use of the medilim, but ' ^ 

decided to save^ money by not hiring anybody wit.h the ^speci- ^'^'^'-^ 
fic responsibility of providing prograramini^, serv;ice a"nd ^ 
assistance in learning abodt the system. The result was a 
very limited development, of expertise at the sit'e. 1) The 
staff who_ were" using the PLATO system had little free time 
' and felt no r«^onsibil ity or inclination t^ prograrrf for * 
others. 2) The undergraduate programmers, had little feel 
for teaching, little time ^available* for hire , and high turn- ' 
over. As a consequence, at le^st'one minircourse slated for 
implementation on the 'PLATO system was put on .other media 
because no dependable, rapid programming aid y;as svailatile^. , / 
The CBE center was used rather jcasually Cor learning, how to 
use another^ language and for recreation, was used for ^ • . 

actual CBE training by, t>hose few instructors who had wiritten 
lessons or who knew how to access materials written hy • 
others/ . . . ' ' 

' : .^^^ { ^ ' ' . . ; ^ , , ^ • 

•«^Case 3B1 .5- -""Insuf f icient referenced and no tra ining ' program » . 



^hprtly after tji? implementation of the PLATO IV 
system, several educational dnsti tnjtioii^ sent full-feime 
instructors *to CFRL to lea ran -i^he TUTOR language and to wr^te 
lessohs. No formal training program was available, no^yvJ^s 
any printed material available for r^erence. At that 
^time there was also a Shortage of terminals, so^fftlit 
traijae<es coulA not always g.et one wherf they needed^ Tt . .They 
had^ to share fjcilit'ies with experienced authqr^who often 
gave them personal help- but also somet imesVlef t xfhe^*tr^inees ^ 
with the^feeling thatAhey were ititru^ing. | Because '^.tie 
staff at CERL aj^d' notfyet understand th-e need to {57^feVide 
training and consulting supporiLXor outsider 3 *wfho were 
novices , they largely ignored this group OT would-bfe 
authors/ The. person, who ''was ^s>^ghed to glv-e 'ihis^.auppor't ^ - 
had other full-time duties and %buld norV devottfythe tUne 
needed to teach them. • Some dociMervta^^ of the la,hgu9iie 
was available on line, but it was intended to be Jised^as a 
reference for . experienced authors.* It, did *not fulfill the 
needs of the^e beg.inning authors. They felt growing**f r us-' 
tratibn at seeing things that they wanted to progrtam bu6 
were unablfe to. The^ result was tJi*^ at the end of the 'y^ar ^ 
little useable maferial had been produced and these indivi* ♦ 
du^Ls were dismayed at bow long it- took to prepare lessons. 
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^'^•jL.?' Lesson designer,, programmer relationstii^k 



Case 



3n2.1 - ad equate pr.e-ii4lanninR ,by desi^eV 



The leaders of one project were experienced in CRE les- 
son design, subject giatte-r ^ c?»si. teaching the target ,popula- 
-tion.. When they decided to use the team approach to l*esson 
development and .hir^ programmers, they made inqQiri^s ^bout 
^ poterrtial problemsmn thi.s arrangement One of the mo'stf' 
.commonly cdted wasWhe "fact • that* some Lesson d^igners 
-assigned a task to" programmer without "e-nough" careful- 
consideration. After the programmer had invested a consi-, 
derable .amount of time preoaring the' lesson 



•eoai 



^ the desi"gner 



wasn't what he w'anted after 



'wapl<f frequently decide tj 

•all, and would scrap the ^sson for another plan. This was 
bad for morale^ programmers f^lt the'ir time hV^Seen wasted' 
' This director therefore madf it a. policy to try not to 
a-ssign a lesson for programming uintil.it was carefully 
•thought out. Sometimes the lesson designer did a few dis- 
plays first to see what it would look lj,ke,or to ilrm^wtli il'j. 
what he wanted, the result was that lessons w«re seldom 
discarded and the prograrhers . dicj not; feel they" were wasting' 
^time. • On the rare occasions when- the lesson* w^s not suc- 
cessful during student trials.,' an* had ' to be revised consid- 
erably, the programmers understood and mdrale' did not • 
deteriorate. , . • •"' 

« 

rase 3B2.2 - ,Profes'sional partrtiifie' programmers ' >)'-'''• 



On? group hired student programmer* on .an Ji<Birly basis. . 
They found J|?fiat this>;as hot satisfactory b'ecaus^there was- 
a high turnov|r*\Q^^r.sorinel.- -"THis meant considerable' time • 
had to b4 sp^t. orienting , riew'people to the particuiHr plans \ 
• and conventions bf theiV project.' Moreover, s^dents who 

■ worked leBs than -hdlf titiie sometimes' founSPthey ]did not have 
♦•enpu*h time to fulfill all of \he c.omrtiitments t/ey'had rfi'ade.' 

Studies and other- activities 'took priority ov&/ their pr»D- * 

gramming ^job. The deci"sicrft wa? made to hire *^iv)fessional" 
.programmers:, thafe is, people who considered this'work as a 

job,, and who coJld devpte-at least half tiore. to*. It, but *• . 
,prefer§bly-57^-^time. They -stayed with the project longer'- 

and were mor^ pr^uctiv'e 

* ^ ^. ' ., . - 

3'. Difficulty adapt ing-. to CRE. ' •.. 

■ Case 3^3.1 - CDE d if Te rent, from cJassroomX 

Hew authors who ha/^cbnsiderable classroom" experience 

found it difficLflt to adjust* to CRF.^ th'ey triecj^to model ' 

. thelrf" CBE lessons on instruction presented in the class-room 
' . 0t - , • - * ' . , 
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They failed to take into ac.count the differences ^between ^\ 
these two modes. * . , • * ' , • • 

* In the classroom th^se instructors' used the di^^,very ^ 
approach to ^learnirig, abiding the students by • aslcia# pertt^ 
n^m questig^ns. *• Aithmlgh. some of the students could riot 
answer the questions tljems^lves, the brighter stutlents did. 
Thus, the teacher did not^ave to provide, the answers because 
classroom* .interaction^^^abl^d the slower -students to learn* ^ 

^ from 'faster one^'. X^^K'the GBE lessons, however, . all oT the 
interaction Was b^weerr student aad lesson. ' Therefore, more 
^^sson guidance wiW necessary • rn , using the'CBE system than 
in *the classroom. Con^^derabre time was wasted writing - 
unsuccessful lessons wMch provided inadequate gui^d'ance 'an4 
feedback for the studl^nts. When the authors observed, , 
student tVials, Xh^y found the studer^ts opeftly. rebel libus 
^The aiithors^ d'ec.ided" to .relise xhe. le'ssonsnBfeiey provided ; 
,more explic4.t ""instructions and adapted thlel^Bfficutty^ level 

: to 'tbe needs'^^of "the students. ,t > ' * - 

Arrival 6f staff . ; / ■ • " 

Case '384. 1 - Staggejed arri^l ' ^ ' 

in preparing for a large pro ject-oriented^developmjent • 
site, the' ma jorifty 6^' the authoring • sta-ff wefpj^given • 
training- at CERL and then sent on t^ a^gi^ven sdte 'to write 
lessens. Six ^ months later a' second group of stajff- '( who 'had 
been selected »earl ier, ^but were hot then , available) arrived^ 
at this site/ The.'author training *for the new authors was \* 
slightly re-oriented to a^d juat. for what ^ were 'percejkid by . 
the trainers to "b^ weaknes'ses in the backgrounds on^the ' ^• 
fir^t' group- of author!'.. In particular , "instruction®! design 
^ training was^^emph&siz^d ... When the second group /^ta^ned * to 

the • pro ject .site, they considered themselves mis^ionar ie^s tcJ. 
O:h03^' who ^ere already established. 'ConvferselVJ tbe^ old- 
timers had already formed' ideas about how to do things* aild : 
considered t|ie second group novices wHoat, pplnions should be 
taken lightl^jf. Furthei5j^6re , dectsionabnade.^iA the first ^ 
part of, the ^^^t. we're hot. not always explained to^ the, 
second graup. Friction , resulted and th^ second 'group failed, 
to become ^^ssimilated* in]|P^ the first, * . ' 

I ^ , ^ ^ ' \ \> ^ ' ' . ' ^ ^ ' ' . 

5. Job security ' ^ ' ^ * ^ ^ \ 

C^se 3B5-. 1 - Low. morale - . . 

In at least three case.s, the*, staff knew that 'the pro- 
ject was scheduled to rwn out of funds. .The ' director s^ gav'^ 
no; indication of whe/trifer further funds might be forthcoming ^ 
or whether other projects might employ th*m' when the current*^ 



prpjec^ended . Since the staff did Hot Knbw whether their 
jobSi^i^ld still be available, they lost motivation and 
wastiia a "considerable* amouH't of time. Often*, project ^ 
reports and other deadlines could be achieved only fif the 
entire staff worked up to, the laat^d^iy of the project. In 
some cases, when faced with the prospect of impending 
^failure, many staff members obtained x>ther positions well 
before the end of the project and. left early.. As' a result, 
projects Wer^ completed weLt beyond deadlines, or in a 
slip-shoct fashion. *^ ♦ ... 

C. Models of Orggnizatioh 

Mahler ja^nd his colleagues (1976) reporVe'd that each of 
the 16.^prpiects they interviewed had a unique organization^ 
Generally, these authors found four broad categories for ^ 
classification. These were: independent developer, a . 
coHea'gueship, a lesson . desi^er with programi^ng assis- 
tants, and some coftbination W these. Matiy ^or the groups 
modifi«4 or changed their organizaft ion in resLpnse to exp||^ 
rience and to the shiftilig needSrOf the project^ No one 
plan was good for every group. Each had advanta^^es and 
disadvantages. . ' 

Early in the development of the PLATO system, authors 
were mostly, professors who^^arned TUTOR. . Some of them 
became pfoficie>nt progralnmers and produced lessons readily. 
When they" hired new staff members the new staff, too^ became^ 
independent. authors, all liked this system because it 
enabled them^to experiment with a wide variety of teaching 
.techmiques and ways of using TUTQT?. It did not force them 
into a predetermine^ lesson structure, but enabled them to 
observe results and to take newljc-geined 'information into 
account for succeeding lessons. ^ Other professors found th^t 
^ they were unable to produce Ijftsons as* rapidly as they would 
have liked — for^xam^^, becalise of difficulties in pro- 
gramming efficiently— and hence hired staff whom they could 
diV-ect toe do those tMngs they themselves woy|.d do if time 
and efficiency were not important. 

Following the export of the PLATO IV system from the ^ 
university environment, "instant" versions of the first 
approach were, tried/with varying success. Far example,' 
s^veijiel staff were hired and set to work u^sing the inde- 
pendent approach described abovf Without the experience 
built up frSii several years of exploratory use of CBE, these 
efforts weje not overly successful in* creating entire curri- 
cula; The authors' independence and lack of experience - . 
'resulted in wide variatioi^s in lesson styles and quality. 
^ The* shirt to a -tearrf appr'oacjj^ met the neec^s of some 

; project* but it was abandoned or modified by others. Its 
^success depei^ded on the* qualifications of the staff anff . 
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the go^ls of the projecfs. In >three different projects, 
when pr6gramraers were hired to code for lesson designers, 
th^y became overly creative. They embellis-hed and i-mple- 
raented'the lesson in a- way that was neither specified nor \ 
^intended by the. designer . / 
•* In case^^here the' programmers' were part-time, ^graduate' 

students, sIBheduling problems resulted in inef-Plcient pro- 
duction rates. Some . directors decided ^to return .to indivi- 
^ .dual authoring and to depend on. consultants to help them 
with particular problems. - 

In two -projec ts , 'a'.team approach became pece-ssary 
because of the^ varied * qual ity of the lessons or the distri-* 
bution^of talQnt within the ^^authoring s,taff Both team^ 
produced adequate^ lessons within th|^ allotted time. Fatftor*s 
contributing to this success were the prespecif ication of 
procedures, experienced management, and ^he dec ig^ion ' to^ . 
limit the instr^ucl^ional strategies.^ A disadvantage .bf ti^he 
team /approach in one incident. Was the disproportionate 
amounts of work by the members/ • V \ 

Individual authors had a perso-nall investment in^ their 
lessons. They were of ten .„^ef,ensi ve abpuV ttiem and reluctant 
to accept ^dvrLce or 'make revisions. Use of the team 
approach reduced this probler# /ince each; per son • S ^^take" 
and involvement in the lesson was no t. 30 ;grea t 



1. 'Lesson designer with ^programmer ^j53^isirfht • % 3 -« 

Ca^e 3C1.1 - Programmers too cregttiye^.M ) ^ 

A prpject heajl was a conten€^^e)ppert -and lihftA^^sity 
instructor. He learped.the TUTOR vlangyafee, but felt that 
his time could be mofe profitably^ s^xent if he designed ^tjbe^ 
lessons and hired a prograiwnef to^ c6de them/ However^, tne 
programmers W6re carried^ away^, with personal* oreatiVl-ty a^d 
felt compelled to contrtbu^;.^. xo th^e lesson. ^Each the. 
programmers hired created ' lessMfe which Here not as intended 
or specified, by the des^igopr-. /Sometimes, they ^embellished" 
the lesson with graphics- and/ahimrations 'that were fun tp . . 
create. The problem was that*^ t^he'Jfc dijjl n<^- improve the lesson, 
in fact, the students became ''f rustrifted ewhen they had to 
wait the few M'conds unti^l the^^rvl of the graphics before 
they could prqceed with the. lesson*. Sometimes the -pro-, 
^grammer changed the, pre*s|jntat ion sli^t'ly^'but the total ' ' 
effecj^ was different ff^i^^^that i^t^nded and th^ lessoYi Was 
apt to fte unuseable. , • . ^ 

Case 3^1*2 - ProgrammJcs too creative (2)*» ' 

The direc^r 01 a large curriculum project hired staff 
members who w^e experienced teachers and/or lesson 
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designers. But many of them felt that theih programming 
skills were insufficient to implement their ideas. So the 
director hired -programmers with ng teaching experience. 
These programmers tended to be too crfatiye by writing 
-lessons that did not conform to thl'^di^ectiVes of the lesson 
designers. The r^esulting lessons .frequently missed the 
teaching objectives and/oft the intended level of difficulty. 

'esse 3C1.3 - Graduate student programmers: scheduling • ^ 
bo.ttlenecks . ^ 

One curriculum development group hired graduate stu- 
dents as programmers. Sometimes this proved to be a bottle- 
neck in lesson production, ^hey scheduled their programming, 
duties «round their courses- and studying needs, .which meant 
.they often planned to program at ftightvor on weekends, jf 
However, at those t^mes the system 'was sometimes , /down . / 
Furthermore, Senior staff were not available to serve as 
CQTisultants.' The students who were on assist'antships we^e ' 
en'title^ to all university ^'acatio-ns. This presented a 
particularlj^ big problem when the vaca,tion was.B month long 
break between semesters. ' ' ' ^ 

2. Jnstructor with programmers 

Case 3C2.1 - Inadequate guidance' 

* An admin-istrator picked parti'GUlar courses in his 
department f^or development of CBE materials. , asked each 
instructor wBo was 3--content/expert , to be in, charge of hts 
own*subj*ect matter. A programmer was assigned-to each pro- 
fessor to carry out the project. The lessons were variable 

*in quality. If the ^bfeg^or provided guidance in instruc- 
tional design pr if the coder had teaching ejcperience, ■ the 
le?sons were good. However, some professors just turned 
o^er content information, and programmers wrote ' ineffective 
lessons . \ ' ^ ' _ 

Casef3C2.2 - Absentee le>«tyn designers ^ 

A professor who was interested in' using CBE, .but who 
did^not have time to become acquai-nted with th^ mediuirr, 
hired a programnfer to develop lesson^s for him. In the^ . 
course of a year, the programmer (a ^radu^te student in ^the 
subject area) acquired an understanding of appropriate' uses 
of the mediu^but was frustrated in attempts to. change the * 
direction ofWie curriculum being produced. Hi*s super- • 
visor's onlyWontact with CBE occurred during monthly 
visits to view the latest materials. Suggestions for design 
approaches tended both to underutilize t^ capabilities of 
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the medium and to make unrealistic derflands on the programmer 
by insisting or\ specific effects that* were very difficult to 
program (and which v<ere often 4)edagogicall^ equivalen't to 
effects which were easily programmed).' ^ . ^ 

In -another subject area, a ^professor with past experi-- 
ence in design of individualized instructloij initially . - 
hired a programmer because he did not want to "waste hrs^ 
time with the jjetails." This professor spent a great deal of 
titne reviewing and modifying tVe lessons he designed. He - 
found . it . very f rustrating to have to work through 'an inter- 
mediary . (particulary sinc'e he could see how easily, most , 
alterations were performed). Fi-nally, tjie professor decided 
V that it was much more efficient for him to learn to program 
directly himself. He ultimately produced many hours o^ 
effectiv-e instruction for CBE. ^>;. 

3. Independent authors^ 

* Case SC3.1 - Sugces^'- 

In at^ least three cases new authors were graduate 
students who had teaching experience and were subject/matter 
experts. Even though they had no^s programming experience, 
they were ab^Ie to learn programmirng* and to write useable 
lessons. Part of their success may have been due to the ^ 
• * fact that their early affiliation with the project was 
-similar to an apprenticeship'. They tiad the consultative 
Support'of a project, director with expertise in all aspects 
of authoring. Furthermore he understood that they needed 
some tiffle to l^arrt by trial and error, 'part icularly in the ' 

* early phases. ^ ' 

... - * • • 

Case 3C3.2 - Shift to hiring programmers 

One group began by having each author design as well as 
'program his own les3ons. The QO-leadeVs found that^they 
wer-e unable to maintain a perspective -of the higher level ,of 
organization of the curriculum. So they dec ided>'^o central- 
ize lesson" design * and make it the respon"s4bil ity of just two 
people. Programmers were hired^' to carry> out the rfetailsS^ 

• The- result was moxe efficient -fesson production and, 
generally better products, 

Case 3C3.3 Shift to team • ^ 

At two sites each author was wholly responsible for^ a 
lesson. This responsibility consisted of planning, writing, 
and codjlng the lesson. • The. result was'that lessons varied 
greatly as to i>nstruc tional approach, physical characterisr 
tics.<cind quality. Also since the authors* experience and 
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talent varied, some depencfed on their colleagues .f5r extra 
help,. Thus,. -for exampie, one of the authors found himself 
burdened. not only with writing the lessons *stgned to him 
. but also with helping colleagues ,who were having programming 
difficulties. Irf one instance, when a senior Author left 
unexpectedly^ the project foundered because nobody was^ 
available to help the less-experienced authors. Both^ pro- 
jects shifted .to team *appr6aches in order to make better use 
of the available talent. 

^\ Team approac^hes * * ' i ' 

Case - Uniform proc^Sfrure provided 

• - - . , ' '■ jf 

At. one -site, the varied quality of lessons pr-oduced by 
irttfeoendent authors led to the decision to establish a uni- 
f c5rnflprocedure of lesson development. This inclxided not 
only tasks that were necessary for lesson development in 
general, such as writ'irrg-vjDbjec tives , criterion tests, and . 
peer lesson revi|w,"but al^so procedures for creating* 
'instructional siTrategiQS and perforrning formative^ evalua-'^ 
tion. ' . ^ ' - 

, This decision allowed for and was folldwed by a 
formal divi'Sion of iabor. It resulted in more rapid lesson 
^rodi^tion, and lessons that were relatively uniform wtth 
respfNct to strategics and quality. ' . • . 

^ ^ ' 

Cwe 3C4.2 - Standards prespec if ied^ 

^ , At one military site it was felt that the staff avail- 
able for author ing * did not have all of th,e necesary subject • 
matter expertise for developing lessons. Previous exi)eri- 
^nce at that site, using independent authoring^ had' resulted 
in a wide range of lesson quali^.. Some of the^lessons were 
unnecessaYilH^elaborate for meet]>hg^the limited objectives 
of the program. The decision was made to organize three- 
member teams , each consisting of a content expert, an 
instructional designer , and -a CBE. expert! The a^Jministra- ■ 
tOKS' also specified standard' strategies and techniques^, and 
a lean approach to lesson design. They ia-llowed no new^ 
teaching strategies and no^^-se'of new hardware.* As a 
r%sult, adequate lessons were produced in the allotted timef^ 

Case 3C4.3 - Preliminary guideline and analysis provided 

, At one site a system^ for less.on envelopment was devised 
by one group of personnel'. Authors in the CBE project were 
bequired/to follow this process ,. although they were' not 
involved in -creating the pla-rf^ It was antic ipated tK^t CBE . 
lesson developmetit could be speeded up because t^e initial 
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guidelines, analysis, and ob jec tives^had already been 
prQpar^d' for them. The creativity of, the. CBE authors was 
lirpited by the c?ontj^ints set by the systems developers. 
The CBE authors dicMiave the advantage that they did not 
have to start from scratch^ in designing lessons. They 
found, however,' that *Ah^ components of the process were 
inadequate for preparation of CBE materials. Ae a result, - 
fnore detailed 04jtlines of the lesson, content were furnishetd 
to the authors* In addition, subject matter specialists 
were bro|ight in to lend their expertise. 

Case 3CM.U - Group brain^tormin-g 

' One group decided tp formulate initial ^versions of 
lessons by^ group "brainstorming sessions." The attempt 
failed. It was difficult , to get' autrtiors to implement ' ' 
session suggestions. Authors felt this system jeopardized 
their autORoray. % . 

• / . • 

• , D. Lessoh Design^ ^ 

^ ^ r 

Of the many factors in lesson design that require- 
decisions, three are represented in the c^ses cited below: 

(1) the use of standardized ys . free-form 'lesson designs; * 

(2) the use^ of minimum standards- and guidelines; (3) the 
degree of pre-pHanning needed before beginning on-line 
design. , ■ ' . 

The "goals of the project .related closely feb the deci- 
sion about standardization". * When the goal^was to^ teach * 
minimum Ijasic 94<illfe, (often 'found in a military environ- 
ments), the objective was to train the .students to a Just- 
adequate level of 'proficiency. Such le^oils . put* more 
emphasis^ on score than on helping a studfent reac+i a very 
bigh proficiency (Klecka, 1977).» Such llssons often used 
standardized formats and strategies to achieve their feoals. 

oh the other' hand, varied tecfiniques constituted an 
appr.opriate approach where one of the objectives was to -try 
to determine which .strategies/wer? most effective in terms 
of student performance and acceptance, or when the objective 
was to provide breadth of^ experience. Authors of lessons < 
with these goals discarded standard approaches in order to 
te%ch ^students to the fullest extent. CompWed to lessons 
with' standardized formats^ these lessons varied widely in 
quality, from truly excrellel^t to essentially useless. 

In order to expedite student ^interaction with peri- 
pheral equipne/it , 'data collection, and lesson debugging^- one 
project set minimum standards for all lessons. In at least 
two projects, decisions were made to standardize the pro- * 
gramming:. certain va^i^bles were set aside to hold» 
information that was necessary for overall curi^iculum. 
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management. When one director of a large project failed to 
make such plans, the results were chaotic. ^ - 

Th.e traditional systems approach to lisson design 
demands complete prespec if icat ion of goals and methods. 
' Some, authors of PLATO lessons ignored, the sytemS nffethod and 
use<l art artistic approach. They created lessons fnich* as an 
artist 'creates^- a picture.* No hard data are available to 
evaluate, the effectiveness and ^production time required for 
lessons done this way. Many experimental independent 
authors, who b4gan with the systems approach, cast* this 
bethod aside ip favor oT an' evolutionary method. They often 
found that when th-ey translated their plans to the terminal, 
many details, had to b.e changed, and the careful planning* had 
been a waste of time. .They preferred, instead, to pre-plan 
or>ly the objectives and content of the lesson. Then they 
designed the lesson gn-line and revised as necessary- as the 
lesson evolved"! ^ ^ ^ 

1. Vajried techniques vs. prespeci.f ied strategies 

Case 3D1.1 - Varied, good 'results* ^ ^ 

One director of a university project decided to try out 
a variety of pedag(>gical styles. His goal was to try to 
determine which as^)ects of less^en design were most important 
with respect tb student acceptability and performance. He 
was unabl^ to' find any differences*. ..The two things , that did 
affect lesson quality were the availability of "help*^^^ 
(supplementary explanations .that could be -obtained Jxi 
Cequest) and good answer judgiprg. That is,. the,^.5irtJdent 
wanted, to be^judged right whea his answer was correct. He 
did not want to be a "mind reacfer" and try to figure out * 
what form of the. answer was acceptable. 

Cas^ 3D1.2 - Var iedj^poor results ^ . . 

In two *dif f ef'ent situations the decision was made to 
allow authors complete autonomy, in choice of instructional ' 
strategies and presentation techniques. Although well , 
versed in the subject matter, ^ the authors* were relative 
novices in the 'areas, of TUTOR programming and instructional 
design. In one of these two cases, goals* were not clearly 
specified. Thje resulting lessoas in both cases varied 
widely in quality as well as in style, 

2*. . standards and guidelines ' 

Case 3D?. 1 - Standards ^ , . 

In an early phase oT a large /curriculum development ^ 
project, a considerable amount oP-time Was spent designing 
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individual lessons and refining them rather than tryinp, to 
do the' whol-e curriculum. These were carefully studfervt- ^ 
tested for specific successes and failures. In this way a 
set of basic 5tandarjis was developed. The origfinal lessons, 
along with the standards, served as a model for subsequent 
lesson development. .A considerable number of the paradigms 
were later used repeatedly with different content matter. 

Ca6e 3D2.2 - Standardization ^ 

Several groups decided to reserve specific variables' to 
•hold -specific infopmation. Later when decisions were made 
to implement cjianges in curriculum management^ the lessons 
themselves did not have I'g, be ' rev ised . ' The director of 
another project found that his independ'ent? authors used 
different germinal function keys for di ff erent' purposes . 
The students wei:£ needlessly confused . . Hg decided' an some 
standard uses, an4^ all less*ors were/^vis&rl - to meet these 
r equlremrents . This greatly improved 'the ease of student 
interaction. Later he prepared sMndar,d initial displays 
which authors could easily attach vo new l:e«sons. 

Case 3D2.3 - Na standardjs \ . ^ ^ 

The director of one large curriculum project made, no 
decision about lesson standards or models. .Each author 
wrt)te lessons independently of the others and made little 
^attempt to coordinate efforts or standardize lessons. The 
result was a -fragmented set of lessons' of variable quality. 
The totality of the lessons met n6 particular paals or 
objectives and it/was difficult to^ evaluate their effective- 
ness . ^ 

Epilog . The d;rector decided to divide the authors Into 
groups, each of which was responsible, for" specific topics in 
the riirrji r ijj^nm . Subsequently, eac'li group developed its own 
goals, procedures, and standards. '.Each gnoup's* lessons were 
a'^more complete and cohesive wholec/and could be evaluated. 

Case - Pacing ' • 

^n one subject area, the students began to expect a 
certain, fasl-moving pace of interaction with the terminal. 
If^a problem was difficult artTT required more than 3-5.' 
minutes of thinking before any answer cogld be arrived at, 
the^tud^nt was likely to. complain, and write a note on-line 
to We author' in a file provided especially for student 
notes. The author decided to provide he\p for questipns 
which th^ students found difficult. As 9 result, there were 
, almost no complaints atout pf'Oblem difficulty. 
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Case 3D2.5 - Incomplete use of the lean approach ' ' 

One pro-jeet adapted both a "leafi approach" and mastery 
learning approach to lissson design. The chief curriculum 
designer decidect^J^ teach only what was necessary, since the 
goal was to produce qualifi^tl individuals at a minimum cost. 
However, rather than modify the lessons where student usage 
showed the original 'content to be inadequate, the original 
lesson was retained and- students 'were routed through it i, 
repeatedly until they passed the performance test. As a* 
re'3ult, tfhe^ lessons provided little branching for remecfia- 
tion help and little corrective feedback information. If a 
student failed the criterion test, he had to repeat material 
had already done successfully ir\ order to get to parts he 
had fai'Ied. This was abrasive to students and merde ineffi- 
cient use o'f their time, but it was more efficient in terms 
of authoring tim6. The author did not have to create addi- 
tional material for help units or program branching alterna- 
tives. This lesson design philosophy had one ottier* disad- ' 
vantage. The authors^did nqt explore ^] ternative' lesson 
formats which might have taT<en longer to develop initially 
but which could hgve reduced training tim€ needed;- 

3 . -Procedures 

Case 3D3..1 - EJxperienced -authpr, shift* ft-om systems approach 

.One director initially used a systems approaah to 
lesson 4esign. He found that when he put' the lesson 
on-line, it turned out quV^ differently than he had antici- 
pated. For example, th'e, screen display, was too crowded and 
the essential idtfa was lost*. He decided to do only a mini- 
mum of pre-planning. He only planned the topics tltat would 
be covered and trhe * ob jeJtive^s of the lesson-. All of the 
rest Qrf the lesson design was done, on-line . This proved to 
"be mcj'st eff ic lent - because the author could look at the small 
sections immediately and revise as he went along. Moreover^ 
new ideas were sponfrtf55i>'t»4yy 'generated in the pfoce^s'Cf • 
iteration. It sho/ld be 'pointed but , however, that thbs 
author was very expisHa^eticed in using the CBE Isystem as* an 
instructional mediurttas well as in his subject matter ^nd in 
teaching. 

Case 3D3.2 - Experienced 'author , shift from detailed 
""'^"^^ plan^ning - • 

One director-author began developing lessons b^ writing 
down all deta4rs. After completing 2 or 3 lessons (about 
45-minute instrAc tional seguenc.es), he realized that parts 
of them had to rewritten. After that he decided not 
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pre'plan large sections^ but to design small section/^at a 
ti^e and proceed as in Case 3D3»1 above. , ,^ - 

E. Peripherals . 

, Peripheral equ^ipmerit added' new capabilities to PLATO 
terinia^s, but also brought compl icat-ions / It was r/ot 
alwaysl^ immediately' available and e^x^erienced users were* few. 
For example, touch -panels aririved slowly, and it was qft^eir 
the case that a toxich panel wasrnot - available for ^v,ery 
PLATO termin-al. The directors had * to decide whether or not 
to program lessons to include the touch capability: In 
three casesf the authors decided not to use touch until all 
of the equipment was' available . Since delivery was very 
late, most lessons did not i-nclude touch. The director ctf 
another project included touch in the .lessoa^L^ but held up 
student tria>& until a"ri of the needed . panel s became 
available^x^'''^ A successful alternative to the d'ilemma was the. 
use of a touch simulation until T^trels were available. 

In many subjects, teaching is greatly enhanced by the 
capability of superimposing terminal-generated characters on 
slides. In order to implement this technique microfiche had 
to be produced. PLATO produltion was not a quick -or easy 
process. The attempt to shortcut this process led to.» 
disaster. ' - . 

The audio device was still in a prototype stage when 
one project director made a commitment to use it exten- 
sively.- J^oh of his staff's time was devoted to testing it 
an(} making 14 easily available.- This .greatly decreased the 
time the staff could spend on lesson development. Other 
project leaders decided not.to use audio because trial les- 
sons demostrated it .was poor quality and unreliable. 

1 . Touch panels ' 

Case 3E1.1 - Not bsed until available, 

'in at 'least four cases touch panels arr4ved Slowly. 
Not all terminals had panels when the authors . were planning 
and developing lessons. If they decided to Incorporate 
touch, the lessons could not be used pn some terminals* If 
they did not use touch, this *nice feature would hot be in \ 
the lessons even when panels^ were available later. At th^^ee 
sites, the director decided not to use touch' input \^tfKil all 
paii^ls had af rived. Consequently^ few lessojis incorpoVated, 
this feature. At a.nother site, the to^fch panel -was i^luded 
where appropriate. However, there were no student fervals of 
any lessons until touch panels were available. 



52 



Case 3E1.2 - Successful alternati^^^ 

• In 3t least three, projects , prograinmers wer6 unable to 
test their touch programs for lack of available equii^ment 
during' regular workir^^ hour s . Thus they liad to wo/k .during 
non-prime time when systems 'maintenance resulted in frequent 
int^erruptions. An alternative plan used hy one group was, 
touch simulation^ Touch wes incorporated where appropriate 
and lessons could be used even without .touch unttl the 
panels arrived. ^ ^ / * : 

2. Microfiche * . . ^ 

Case 3E2.1 ^ Shortcuts didn't work ' .J 

Francis (1976) has provided a derailed account of the 
difficulties encountered in the production of microfiche. 
One decision that always ended in disaster was the attempt 
to expedite production by shortcuts oX'one ^sort or another. 

3 . Audio . ' / , / ^ \ 

Case 3t:3.1 - Time Jlstraints 

Th^ d^'ector of a project committed himielC to ^se of 
audio when it was in an early prototype stage. It was .to be 
an essential part of -instruction In order to insure 
availablilife^ of the audio equipment in sufficiently 
reliable'form, much of the group's time was spent testing 
hardware and providing service' software so the audio could- 
be used, easily. This severely limited time' available for 
needed course development work and Tor field testing of 

•materials. ^Late delivery of hardware also limited access of 
students to the material.^ -Total productivity for the grou^^^ 

• in terms of hoijrs of instructional material per hour- of time" 
expended wa^ over 2000hr/hr. \ . / *; 

Audio disks could not* be reproduce^l fast' enough, to meet 
the needs of all users. ^ .Lessons could not b^*.used without 

^the audio. The intended population was ^dismayed . Also, 
slow reproduction siphoned Off s^^ff time and limited the ' 
number, of lesso^^s that could be prVduced at a much needed 
higher cognitive level. A 

Cas« 3E3»2 - Conscious decision not to use auc^o ^ - . 

> . At .le-ast. four project directors f^rt that.^an audi</" 
device would be very desirable foh their CBJLJessons. In 
some^cases their students ^were poor rea^ders. In^thers, 
sgund such as a , human ' hear tbeat , was an essential part of ^ 
^ instruction. . The^ wrot# mini-lessons to* test out the - *- 



quality and reliability of the audio device of the CB^ \ 
^system. ^h§y found the sound* was poor quality, partly, due 
to^excessive bafckground noise. The audio was also unreli- 
dble:' soTnetimes messages were wrong and occasionally they 
slSply did not occur, /Consequently the directors xTecided 
not to develop lessons that depended on that audio ^device • 

Castf 3E3.3 - Dabbling \. / 

, In several instances, authors "dabbled" with periphers^ 
equipment. For example, they included microfiche in just 
one place, or touch responses in a^few displays. They 
mistakenly thought they were only supposed to 'use these , 
peripherals when absolutely essential. For example, they 
were to use the touch only in instances' ^here it would be 
very difficult to have the student make his response by^^ 
pressing a key. ' In every case, the results were.b'ad, The^^^ 
author did not take enough time to understand how to use the 
^peripheral device 'effectively^. Consequently, the affected 
parts of the lesson -did not .work properly. * » ^ 

F. Implementation and Integration . of CBE 

It was not th*e case that PLATO lessog,* we'r^ ' 
automatically -integrated into existing programs. Charac- 
teristics oj successful integration Were: (1) lessons were ^ 
easy for the injS^ructors to use; (2) information about ^ 
si^jdent progress was- provided; (3)* instructors ' ^ requests 
were given careful attention anad consideration even^'' though 
they could- not always be met; and (M) a proctor was av.ail- 
-able in the classroom. Integration with traditional 
instruction was facll itateci *when the , instructors were 
involved some way. ."^^ ^ * 

Major difficulties included instructor resiatan*65^^ 
CHE and resistance to using somebody else's les3ons'. ,Uni- 
.ver'sity students resisted doing the lejssons when they did 
not consider them to be 3 regular part' of instructibn,^^ like 
lectures and laboratories. Some lechnj.ques ^ that Jictlped 
alleviate the problMi were: (1) scheduling PLATO sessions in 
the time table; (2) giving extra points' far completidg.-les- 
sons; and (3) includfng some questions from, PLATO lessons: 
on the hourly tests.' "Students also ob jec ted. wheri the tfer- ' 
minals were far from their uspal claiSPOom buildings. 

In some inst itTutions one classroom /bf terminals was* 
available for jn.any. cj^sses in' many"discipl;lnes. . Stfhedulirjg 
problems were, par];icuWrl^ difficult when^terminals were' . . 
assigned for evenly-distribijted use Uy each class but the 
-lessons required poncentratel , *da"y-after-day usage. ' 



1. Fadtprs. that co'htr.fButed to successful implementation 

' ' ' ' . . ■* 

' * Case- 3F1 . 1 - - Make .it /easy for the instructor to Use - 

The initial plan in on^ project was to' give the • ' 
^ teachers- control oyer sequencing lessona. The hope was "that 
^ ^Ij.^f^.^"!^ 'ericdur,ag'e. them to integrate - the CBE lessons with 
•the rest of tlieir torching in^-this s,ubject. However this 
, invol'ved a 'fair ampunt; of attentiojr to detaij. on the paVt of 
. I^he t#achft:s "and was notjg|a|^ to implement. In another pro- 
ject, the director devel^B|M|i'" considerable liody of lessons 
for his discipline antl wa-^Wfer for others to uag/them, 
- . made^'t .ea^y and cot|»ten i e.n t to' use the msons, 

.even includiijg»on-line„ tests (although Ui.ery were npt called 
I tests). The more convenient he made -it the" greater was the 
general acceptance. . ' • • , ' - , 

. ^ Case 3F1.2 - Inform the inst?rucfe"orsft|||?out student progress 
^ ' . - r ' 

■\ '• cases., CBE terminals were part of the. class-c 

. noom equipment. Some students did CB£ lessons v/hile others 
.w§re ^eng^^d 'in di-fferent activities. . The instructor 'was • 
t.bo busy bo'; observe the students when-.thev were using the 
. CBE lessons. let they "wanted to know how their students . 
; were',doing. • Initially, the project director "decided to pro- 
vide on-line information abptit ttov.each student twas pro-' ' 
, tgressing,- anfJ in great detai-1.'^ However, thereiv*as too. much 
, for .tltfe^ instructor to read and too much data^ fdr him to get 
^Ua' geferal 'i^eaxp^ how t*he. ?.tude'nts"'were doing. \ The 
^jjBnst¥|ptor Simply did not use the' detailed, infofmati\)A vhen 
^t wa^available, only^ ih ttiis form. In .tJjM project, CBE' 
lesson? wer£ just one aspect of classroom instruction . The 
^ project ^ectorXdecided to make, two qhange's. Data 'made 

„ available to the, insti^uctors^ wa^ very simple and referred' ' ' 
ft- .only to key lessons." ^Usually .all of. it could be presented 
m, in one screen ^iUsplay. • (2) A printed copy of the data -Was 
. madef &nd.'giv.?n to the instructor. The resul* was .that -they 
^T'vTere. able to tell very quickj^ -how well-'the students were 
;. do^ng. -IT ^they'd id' fiot have time to look at-the data' 
•V o/i-l,ine, tJiey cbuld^nd did take the printed copy home with 
7 . them^ and looJ< at/it at -^^Jh^pir oonvenience. 

Case ^1/3!*'.-,Listen'-tjD instructors . ~" " ' 

' ' . . . " . ■ . . 

.Jn two^ large •cujffiajilum- projects decisio.ns- about ■ Uie- 
.Jes-sCrf .content ' wer^ m|pt at -,the development- center '. In the 
rli^'^*^^^^^'- ^"stpuCtqrg w^ne riot overly enthusiastic ^beut 
CEE.. ThoSe^who use.d t.he matre-rials T re'queotly made requests' 
for^jjifferept on' .add it ion al lesSons as>ell as pirtioplar ^ 

kinds of • fn^Armafci n.n "Tha w,^,- 4 y^TT . . _ . x-i. . _ . 



kinds of • fnY€>,rjiation'. ".Thg authoring/^aff>was .often riot 



» 
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able .to meet most of these requests.. .However; whea the 

staff tpok the^ time ^to listen to .the ipstructors and ta]^ to ^ 

^hem, the instrifctprs * •'att itlfdes improved,' 

< 1 . { • 

) i 

Case 3F1#4 Lesd tirae for instructors . . 

vfel^lfcitary school teachers wefe very busy during the 
3cbool.vear: and" did not sign on to the terminal very often. 
Therefore tljey did not s.ee. many of the, lessons that were 
.developed 'during the .jf^^c, ffdr did they gain' the "comfort"- 
(ir feeling of eade^ withuthe machine that comes from inter- 
acting with it. The .teacher's attitodes toward CBE instruc- 
tion imi^roved with his degree, of *omfort , and his ^attitude 
w|is reflected in the students/ When possible,/ time was 
>providecl during, the summer for teachers to review the 
materials and experience working 'as a student. The result • 
wa6 that they were9more ab'ease using the CBE system and ^ 
more likely to treat Hhe lessons las an integral par't of the 
class. ' Students then picked up this attitude and took the 
CBE lessons seriously. , 

Ca«i3F1.5 - Proctor availabl^e in classroom 

One project 'provided 5 classroom proctor and also' an 
line f^le in vThich students could write comments. One of 
ie^fa(s€*r^ that cor\tributed to the director's decision to 
, ' kee^ a proctor on diit'y, was. that the students had "better ^ 
..faelings" when a 'person wa? arounjl. In fact^ the tone 
^ -t«r comments Ih, the note files was much milder when^ they,0 

^ could tall< about their - comments to a ^jroctpr. 

Another project began to uge -CBE lessons for regular 
crosses before-the lessons were carefully revised. The 
director decided ^to have, proctors tn.the classroom to help 
M^l^ Aire students over known problem spot« in these- lessbtis. 
Hft/ ^'hus 'they*coHjld use otherwise urruseable lessonS until 

''^^thors had a chance to fix them., ' * ^ - - 

^ Epilog . ^^te^ several semesters of experience and cbnsider- 

'V . able lesson revision, they decided to. retain a proctor in 
the 61assroom. During^the' fi>st two w.eeks of Ih^course. 
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# the Classroom. During| the' t irst two w.eeRs ot tn^course. 

; ' ^ this was nercessary t<»;ake care of various beginn£ffl||i|jgf^ 
% ^semester problems. .M^ter that, the proctor was available 



.e to 

help students^ith related, non-CBC difficulties , as well as^ 
to. write notes to the authors-- about pertinent observations. 
He als6 ,€hcoliraged students tp put notes in .the comments 
file. * - . . • ' ...H^v 

Case 3F1.6 - Show the student his progress 

,A curriculum project lmp»lemented a mastery, learning' ^ 4^ 
Strategy in one„ section. That, is, the student had to ^ 
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achieve a preset criterion in order to move on' to the next' 
level .of difficulty. ' -Initially, the student's prog^e5s was 
shown only to .the instructor and not to the student,. fjtie / 
S-tudents thought they were moved from one leston to another 
by some sort of magic. Motivation was poor. The decision 
was made to show the student ihis progress and to exp^tsin why 
be was moving ahead or repeating a' section. This approach 
resulted in better, motivaf ion and c incre'ased instructional 
progress. - ' ... 



2. Ffaetors foj" successful integration 
Cas'e .3{;2.1 - Printed copies q|f lessorts 



I 
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The lesson developers in one^^project were eager to have 
the teachers "integrate PLATO lessons with regular classroom 
lesspns. In order to^encourage this, ^RE personnel made ^ 
wcbr'ksheets that corresponded to the PLATO lessons, often . 
» getting 'e^ectrqstatlc ^prints of terminal displays aad repro- 
^^lucing , them for the . entire 'cl ass . This. Had many ben-efits.- 
Lt enabled the teacher to observe the children if the whole 
class w^s;worl<ing on th^ worksheets at th^ sa.me iime.- 
Altern'atitrely , ^tie could look at their work after they handed 
^"the papers. -Otherwise, he* might not tiave' mijch opportu- 
nity, to observe them studying the' lessen because, for the 
most parj|; he was occupiecT with the re'st o^the class when 
\^any :€^uf .Jchildr^ were taking their turn arthe CBE 

terminaT's. /TlsV the teacher could tfl^e the printed copies 
home and locJk'at tPle lessons at his own convenience. The 
children (liked tKe idea of worksheets related to their PLATO, 
i nstruction • ^ . ' , ^ 

Case 3F2.2 - Involve instr^uctors \\ 

/ / > 

.Early in ^t^e development of each of ythree^urriculum 
"projects, lessons ^were written at th'e -devel^ffment center and 

* brought J:o the participating institution. , Students took the 
-lessons in- a special classroom..^ The instructors had pre?^ 

vpusly attended' some orientation meetings, but were never- 
theless dijBinterested in the CBE prograTil. 'Later in the 
project, the decision was* mad.e to ^consult the instructors 

"^^fore lessons were written Vith the^ object ive ' of coordi- 
Awting th$m with standard classroom materials and methods.- 

./the resiilt^was interjest and cooperation from the Instruc- 
tors. The ^'Stt^d^ts''' attitudes changea from regarciing. CBE as 
an' "extra^' 'to cons^deTlii^^-l^^ part of^thelr 

• studies, ' . . » 
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Case 3F2.3 -^Instructor- control , , ' . 

One project director organized .his ctarriculunt in a 
tree-like hier^rchfcal organizg^tion . He developed a compu- 
ter router wHich automatically Tirade all decisions about the 
"optimal" next ^lesson^ in the hierarchy,* Unfortunately, the 
pedagogical needs-of'the students were different in each 
skill' area ^gnd the generalized decision jn'^ing scheme did 
noti m§e't allof^ these- needs. \ % 

— ' - . / ^ - ^ ' ^ 

Epiipg . The decision was made to change the lupous of con- 
trol . Management control was transferred to the • instruc- ~ 
tors, who set up a daily list o^ lessons, for ea^h student?*-^ 
The results were at good ^s.. or better than those undej^ the 
automated, system. ' . ' 

3. Implemer^tatio^^_Lproblems ' ^ 

Case 3F3.1 - Not enough * terminals 

in €ne institution many different cours^s'of f ered CBE 
lessons, but there were not enough terminals.^. The decision 
was made to allow e^h student one hour per weeft, during 
'class time if\possible. In one discipline, some of the 
lessons were intended to .be uaed in five or six consecutive 
sessions. Onces^.week was not eff ective and the students 
could not get the impact that was intended. 

Case 3^*3.2 - Resistance to using sometfody else*s lessons 

In one discipline, CBE lessons were prepared., on topics 
that were part X>f the standard instruction in the beginning 
courses. These courses varied sl^ightly according to whether 
they were intended fof majors or as a-service for students 
from otheF departments. The , department head m^ade the*deci- 
'sibn that the CBE lessons should be used to replace lectures 
on those topics : * Instructor s showed different reactions . » 
Orre instructor i^ho taught "majors" resisted rising the CBE 
lessons. He claimed* the lessons were too easy. * Another 
instructor- suggested that this was an excuse, and the real 
problem was that the individual was near retirement and* a 
victim of inertia. An instructor who taught a service 
course fop gr'aduate students use4l|lkhe CBE lessons to replace 
lectures. |ie complained that the, students did rot j|et 
enough out of the lessons because they were not tied to the 
homework. Instructors who taught large* service course 
abcepted the CBE lessojis and used* them to replace one of the 
two "weekly lectu'res throughout' the co«iiB€. They considered 
the lessons quite acceptable becdtise there* was no decrerrrent 
in student perMrmance. 



Casfe. 3F3.3 - Resis.tance 'to. CBE . . ■ 

One group of individuals 'in a university department 
developed* CBE lessons anH macle thetn available to ^all members 
of the dA)artment. The ' older* staff Who w«re fiear retirement 
professel an interest, but never found the time to look at 
the lessens. ^ They refused to have anything^ to do with >the 
•lessons as part of their courses. They did not even try the 
lesson's or observ^-e students doing them. It as posslbLb that 
these people were sifnply not interested in any innov^ions, 
but it is just as likely that they were wary of computers 
and'CBE in particulars 

Case 3F3..^ - Increasing stu<ient attendance \ 

In several- ppojepts, spme students d*^ not consider CBE 
les:§-ons to be a "regular" part of the course. Otherg^burtd 
the classroom pf terminals- was not conveniently located. v 
The" instructors^ found too many students were not ^trying the 
lessons." One project director -decided , to give two extra 
points' toward the semester grade' f*or each lesson the student 
completed. Another instructor imiiniin i iV litmj some "of the 
hour exam^ questions wouXd, be tak^n from inforffl^tion g-iven in 
the CBE lesions-. Both meViods were successful in increasl^ng 
student attendaooe. ^ ' 

Case 3F3.5 -.Administrative dilemma, games 

Onfe ^f the administrative decisions that had to be m^de 
was whether or not to allow game playing at a site. One 
problem, seen fairly often, hgd to do with /'PLATO drop- 
outs" — students who became game freaks at the.- expense of 
their studies.. Another consideration ' was the adequacy of - 
resources for* both regular lessons and game playvng. » In at 
least two university terminal* classrooms the .site director 
or^ganized a^list>^ games for students to be»^able to aecess. 
The top administrators made different decisions in the -it^^o ' 
easels'. » , . , ' 

\ Insane case, the site director set up^alijst;of games 
to make the PLATO system moi'e appealing. TheNproject direc- 
tor decid'ed ^that, this was a pobr policy and discontinued 
game. playing. He felt It wourtr encourage stuj^nts to think 
of the PLATO system as a source of ganjes aVid not .as a ser^ 
ious instructional device. Me also fel.t-tha^ some of the- 
students who were being* attracted to .the .classroom- >/^i"^ ^ot - 
.parvt of the target populatiorf. Their presence was certainly 
not good advertising for the project director wbo was trying 
to justify the ^advantages of CBE. * 

In another case, the site director felt that fames' 
enabled students to get a refreshing o+iange of -pace from. 
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intensive study.* Terminal usafee was completely^restric ted 
i" . so problems with outMd$rs never arose. The top adrainistra- 
\ tors^did ,not object tg the availabiTit^ of games, and^no 

serious problems were reported, ' • _ 
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Form^ative evaluation, that, is, evaluation during les'son 
development, early became a necessary part«of the process of 
lesson development. In fact., ih most ^ ro 1ec ts continuous 
evaluation an integral part of '"leOTon production^ imple- 
mentigtion and maintenance. Edvards an'd his colleagues 
(1975,) have suggested , thst at least four kinds of changes 
t-^ke pl-ace as a progr^m evolves. They alVgued that evalua- 
tion , s^o.uld"'" be continuous because these changeis are contin- 
iY>u^. The changes that ^ acc.ur "in the- development' of a PLATO 
lesson closely parallel jbh'ose .that- the^^enumera ted.. (1) The 
objectives^ of the project ^sometimes changed, as' from 
r&s'earch to applicatioTis orientation. (2) The. program 
changed in character, such as from being s^lf-^cohtained to 
being an" integrated part of a larger whole.! (3) The program 
was established in response to some educational or societal 
circumstances, and**t)^ose changpd. > For. example, initial - I 
plans for -an innovative ciipifrculum were dropped fdr lack of 
fuYids or phatiging irtterescs. .(4) Knowledge accumulated and 
dictated. ch;ange'S . This was true of all aspects of PLATO 
projects . >i ^ . • , 4# 

It was Important for each PLATO project to have an 
evaluator as a^staff member' or a* a consultant. » In on^ - 
--incident, where there was no one person, i'n. charge of data 
keeping, c^verlapping responsibilities and an inadvertent 
breakdown dn communications resulted in a considerate Ipss 
of data. When arv evaluator was not consulted, inappropriate 
statistical techniques were employ^ and the results were 
difficult to interpret and generally questionable. 

Planjiing approj)riate formative , and summative Evaluation 
during the initial stages of project 'development was essen- 
tial. By making such plans, one project was able- to evalu- 
• ate-the effectiveness of what was accomt)lished even though 
they did not meet all of their goals. In contrast, arn6ther ' 
group did. not plan for evaluation^ and also did not me^t its 
goals. As a result they could not ev.en evaluate 'the lessons 
they did complete. ^ - ' * 

In general, data ^were used not. only to assess ^^t^ec- 
tivenesjs but also to revise and improve lessons. Formative 
evaluation includ^'d lesson reviews,' student 'trial s, and 
lesson validation. The impact of lesson reviews on revision 
'.depended on the timing of the .review, the- qUal if ications of 
the reviewer and the nature, of authar-reViewer relationship. 
End-bf-lesson reviews .by the outside consulting staff 
tfff*ected few substantive revisions. 
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• This lack of acc%ptance was surmised to .be due to three 
psychological factors (Call-Himwich, 1977). )• Authors, 
.were subjectively involved and. viewed th6Tr l4ssons in much 
thei sam6 way an artist views his >rork. ,(2X By the time an^ 
author received t^e-^review he had already begun work^oh'a 
new lesson and was deeply involved in it. The old l.esson 
was "cold" and .had been , tucked away mentally as "fy^^shed." ^ 
(3) The totality of so many suggested revisions may have 
been overwhelming • • ^ * ' 

Revie.ws made during l.esson development increased the 
probability that lesson revi-sions would be more than cosme- 
tic. Experienced teachers and colleagues who were' also 
axrtrhors were usually effective reviewers. Subject matter 
ex4)?rts tended to look only at content accuracy. It was 
^elt that univef si'ty-priented reviewers made in^propriat? 
suggestions because they ijid not understand the ^nv ironment 
or needs of the- non-university student. F'arce-to-jTace inter- 
*activfe reviews^ between author afid reviewer seemed to be more 
effective than written reviews-. 

Student trials, were impoirtant .far both lesson'Vevisio'n 
and validation.. Collection oF on-line data, such a-s^time 
and record ®f ^res^onses , en-abredthe author to 'revise the* ^ 
lesson to fit 'the allotted time and to detect probierii/areas. 
It was necessary for authors to observe student t^als in 
order t® note -trouble spots which on-line data did no-t 
reveal. Reliable inform^ition .was <nt>st likely to r^ult when 
the students who tried out the lessons were from the . / ' 
intended' student populatijon. One project made sppcific , ' 
, plans to validate lessons and this validation' helped smooth- 
implementation. Failure of otfter groups to Mlidate* 
resulted .in some. unDseable lessons and maify frustrated" 
situdents. . . ^ • . 

At least four project directot^s • establisJ^ , notes ' files ^ 
and' encouraged* students to make on-1 i,ne "comments about *le&V 
sons. One director said they. were valuable but three direc- 
tors said the comments provided little useable informlatioh , 
and sometimes they were in bad taste. , / 

In at least four projects, the student coulil , request " 
on-line informatiqn about his status *and progr^ess." This .was ' 
partioularly motivating and saved a great deal of' instruc- - 
tors* Aime. In/one project which did not make - such, info^ma- 
tion available, a'^major complaint from the students was that 
they did not know where they stood ih the course. V 

Les's«!>-'ef f ectiveness was measured in different wa'ys . 
Some projects* compared test scpres and/or learning tim^ with 
a ^ontrol group; Dthers did^pre-test , post-test comparisons 
tdl^deterniine gains. Most ' prjo jects used, some^ form o-f opinion 
ques'tionnafre. No specific cases ai^e cited 'below, but it 
was gener^illy felt that .these questionnaires provided the 
project director with .useful insights about the students' ' 
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attitudes toward a particular lesson or the entire CBE 
course. * - 

A. Planning* Evaluation 
1. Need person in charge *of data ^ pl- 
ease - Loss of data 

Evaluation of les^oh effectiveness was a major goal of 
^on^ project'.- the staff did not include an evaluation spe- 
cialist.' One objective was to coffloaret CBE lessons to non- 
CBE. Students in ^both CB€ and control groups were required 
to .take a 'paper-and-pencil quiz at tlie end of each topic and 
to meet a spec*ified performance criterion. .If the ^tudf^nt 
p^assed, he received- a "pass" rating ' and ^was allowed to f 
continue on. If he failed/ he received a "fail" rating and 
had to repe.at the lesson and quiz cyclg agaiil -until he 
passed. The pass/fail data, had/j^eerf ^thosen as one measure 
of lesson effectiveness., .The other measure was. the total ^ 
Jime it took- the student |^ successfully complete the 



lesion.' 

Time data for CBE sUjdents were automat^ally* collected 
by the computer and tr ansf 0«*«45Jto.permanent storage for 
later use. , Times for CQntrO'l stu'h^nts were recorded by the 
instructors a'nd later^ entered into a computer #ile. All of 
the pass/fail data was to ' ha vis been' entered Into a computer- 

^il'e but the computer progr^ was not available in time. 

i^her^ore the course personnel recorded each student's data 
as he progressed and later collected all the data. The.y 
infbrnrve^l the CBE staff that would keep it / Indef i/iitel y ' 

or let them' know if. it was t^jPR destroyed* However, when 
the CfiE staff requested t^e da^, they found -that^ sbme . of 
the data laad been inadvertently destroyed, linf or^unately , 
not all of/ the origlp^l data was recoyerableX This J.oss 
could have been , prevented if the responsibility for data' 
-keeping had been assigned to a single individual. 

•Plans for statistical analyses of data were rnad<? with- 
out consulting a specialist. The. most appropriate /statisti- 

-cal test's were not chosen, and the- results were not as 
reliable as they could have, been. * ^ 

Case ^ih^2^ - The need for Experienced evaluator 



. »To 



me site, there was nb experienced evaluator on t+ie 
staff. ^Tbwever, staff members patiently collected quite a 
lot of data before they started ariialyzing, them. A cursory 
glance at the data showed that they had been -.crudely 
gathered (by non-project staff). After improvements to the 



data gathering, it was founc^that non-CBE instructors felt 



that the evaluation data being gathered could be unfavorable 
toward slow students. So the staff members were lenient • 
toward thenyfand .gave them, "breaks" to make them laok better.* 
Exper^iencetf evaluators would have begun analyzing data 4?nme- 
diately* as well as devising tests artd checks for the valid- 
ity of the data. ; ^ * ' ^ ' - 

2. Need to pre-plan .evaluation 

Case i4A2M - Rewards of planning 

One director selected the target cqurse for CBE and set 
up ,the hypotheses he would test even before authors wfere 
trained. As a result, project developmen.t^moved steadily 
'toward testing the specified goals., Althpugh some. of his 
^data were marginal in valqe, the d^irector wa's abl6* to J)ro- 
vide som^ useful evaluation* 

Ca^e ^2.2 - Shift to formal evaluation plans 

.A site had- the responsibility of developing instruc-^ 
tional materials for. a large portion of a course. Either 
because of the press of daily events or because, of a cpn* 
scious decision, ffS formal formative evaluation program for 
the lessons being developed was instituted. Soitfe lessons 
were tried out in trial student runs, but these feriai runs 
were, haphazard at -best. Peer lesson reviews were^recom- 
mended but were by Ro means universal'. The only constant in 
the formative evaluation of these lessons was the outsid6* 
reviewi^.which were done ^several times for most lessons. 
This service combined with site peer reviews was mistakenly 
presumed to be as goo* as thbr-ough student testju^ of the 
lessons. When- real students began using the lessons, sev- 
eral gross errors appeared in the'm, causing a great loss oT 
confidence on the part of the students. This loss of confi- 
dence was surely a factoV in tlje eventual failure of. the 
project. Second, 'the authors were fdrced Xo revise lessons 
as the. studeats revealed* errors in them rather^ than develop 
lessons for the next segment o'f instruction. This had J^he 
effect of putting <he staff eveii farther behind in an 
already heqvy schedule, ^ ^ \ 

a later time when the prt>^ect> was again reprgan- 
'ormal p^n for evaluation was written and the 
data were collected.* #Jhough not perfect, they 
adequate. 

Ca%e^^A2.3 - Failure tp plan evaluation- 

At 'q^^ sit,e, no evaluation plans were' written ini^^^m^ 
'tia.lly. /-Consequently ,• as the project progressed, plans 
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had to be written and rewritten .several tim'es due to the 
l»jjj< of agreement between various staff members who were 
responsible. Every time' new plans were developedt, 3 
considerable number of lessons *were discarded, because they 
did not .fit the newly devised plan. 

* • • • J 

• , B. 'Aspects of Formative Evaluation ^ s ' * 

1 . Lesson reviews • , / ^ 

*Ca5e ^Bl.1 -Timing ■ ' , * . 

' In, one in^ance, a cc^sulting and support p\:.att wrote 
lesson reviews after the lessons were completed. ' The 
reviews were .submitted ""in writing and delivered to 'the 
author ong^ to two months after the leasons were completed. 
The reviews covered all aspects of the lesson,^ instructional 
and programmihg quality. Although the authors said the 
reviews were helpful, authors made only minor changes^' such 
ms correcting miespell ings . /Although in .one case a review 
noted a' programming error which wouLd halt student progress 
through the lesson, the author .took qo actipn. 

Epilog . Realizing the motivational and psychological draw- 
backa to receiving an exhaustive critique- long after tl%e 
lesson was "complected", the staff experimented with various 
alternative reviev approac'Kes . . Thej^ finally developed a 
plan by which lessons were critiqued in-prbgreiss , or as they 
were actually being writterr. Reviews were ghar^ter, fewer 
changes were . suggested at one time, and^ maj^r problems could 
b^ deajt with before th«y became chronic. In 'addition, a 
new-feature allowed rev-iewer^s and authors to go through les- 
sons "together^" each seeing the same screen display even 
though they. were miles apart. Reviews thus became mor^ an 
exchange of ^ ideas than th"e *^monologues they had once been.* 
Authors responded much more favorab^^#nd incorporated 
50%-75J of all suggested .changes. 

' Case ^B1.2 - Reviewer credibility . 

One individual from the supporting institution reviewed 
lesson^ of some authors before ^he had met them. F^e foixid 
that these authors were not as receptive to suggestions as 
wais the case when he had met the aut>ho'r at some time pre- 
vious to that of the review. Several reviewers found that 
they were more ef f ec ti ve ' wf^en they reviewed the , lesson in 
the author's own environment. The reviewers also reported 

^that they were more^ " sympathetic*' with the author's pro- 

''blems after the> made a site visit. 
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Case 4B1.3 - Reviewers lack experience with target students* 

In three cases, the consulting Staff were university 
p^ple. The authors rejected the consultants '^suggestions 
becase they, felt that their 'Stud^ftts were' very^d if f erent 
from university students. Con^quently the criticisms and 
recommeftdations were not considered applicabl-e. Hbwever the 
reviewers noted that many of theip. sugg,e5tions were like 
those stated in more general instructional -guidelines at the 
authors' institutions. The reviewers perceived that th<* 
authors were simply Osing these' arguments- as an excuse -for 
not accepting the recommendations • 

Case 4B1.4 - ^Reviews by subject matter- experts or peers 

The organization erf information tkat would l^e reasbn-r,\ 
ablfe for a content .expert may be quitAdif f erent frbm that 
which would be. presented to a novice f^fc Gla1s6r^4 Resnick,- 
1972) . ' Peer reviews were, important buK^ could nbt' rep,iac'e 
student trials for information value. ^ Peer leSson reviews 
were sometiroes .only content' specific ''if peers were -not 
actively involved in CBE. • ' ^ * ' ^ ^ 

^ Some project authors did npt like to -makfe th'e lesson^ 
design changes -sugge'sted by subject mattfer and other 
reviewers. ' Changes took time and they felt that if 'the 
content was correc^^^ the 'lesson was good enough. Issues in' 
lesson design Were regarded Is a matter of author whim and 
unimportant for learning. -They therefore requested that 
reviews be limited' to content only and disregarded arw 
design' comments made. As 'a result many fe viewers loroked at 
only one lesson and refused to do more. \ ' • 

Case i*B1.5 - ^acher'S, author-f olIeague$ 

In many projects, authors who worked together reviewed 
each other's lessons in an informSil setting during develops 
ment.. Although there- was soitie"^ def ensi veness^ among new^ 
authors, experienced authors found these reviews helpful and 
sought them from respected colleagues. In the elementary 
situation, teac-hers sometimes added perceptive iiisights for 
lesson revision. . , ; • , ^ 

2. .Student trials , * . * ^ 

Cape ^B2.1 - Pre-plans to valijdate 

At one site the decision was made during initial 
planning that there wbuld be a specific number of trials 'per 
lesson anxl a ^iven number of students per trial. The result 
was a rather smooth impleme^i^tation . At one site, parallel- 
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.lessons were written for CBE *and fbr non-CBE media. Plans - 
were made to. validate rton-C^E materAafs before transferring 
them to CBE. ^ 

C^se 4B2.2 - No va;Lidation • " * ' 

One group- needecf a fixed number '''of hours ^f lessons. 
The project director did not make any plans for student 
t'raals or validation. Students aut'ho'red the lessons and did 
not have the time or ^exper.ience to test them adequately with, 
other stud^jfts before classes were to use the lessons.. The 
results^ was that -many students in -the class were frustrated. 
They aoulel not always"^c<Mnplete the* l^sgpn^'because of .pro-, 
gramming errors, inadequ^e answer judging, poor^ explana- 
tions, etc : ' . . . ' 

■ • •■ A 

Case 4B2.3 - To interpret .data ^ \ 

# • 

Data collection is' an important part of CBE lesson " - 
writing. In one group, developers foun.d that some of the 
data ^seemed unreasonable, so^ they decided to monitor the 
classroom whenever possible. As a result they found explan- 
ations of data which they f)reviously could not understand or 
interpret. For ex'a'mple one ' student did the work for ^ 
another, or € long time delay was the result of a conversa- 
tion with others around him. 

Case 4B2.4 - Authors didn^t observe -student trials 

At one site, authors had been, told that they, should 
observe during student trials, but refrain from helping the 
students unless drastic errors occurred. They were also in- 
informed that it is very d if f icult* for an author to stand by 
when 'the students struggle with his lesson. Some- staff 
member therefore diecided t9'bar authors during trials! The 
only information that th-e authors received was second-hand 
and abbreviated, of course . /^^Consequently Cfiey failed to 
Learn puickly^ from stOdent trials. 

Case 4B2.5 - Inappropriate students . ^ 

In one case .students from apoth^r course- were- used 
This was not^ a .reqair;-ed course for them, and they did not 
feel responsible ' for learning;, the content,, so ^Iney had'3 
casual attitude toward the lessons. The author s- assumed 
that the lessons would be satisfactory for the target stu- 
dents and did not revise the lessons as they should hSveJ * 
When the lntended sjt'Udents tO'04< the lessons, they saw them* 
y^ry differently. They 'could not learn. the rrfaterial readily 
and were^very upset. ^ . ' _ ' # 
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Case ^B2.6 - Students* comments 

In one proj-ect, <iuestionn'air?s were handed out to the 
students to' help tl^e authors evaluate, the lessons. Students 
ch^cked^ oYf-attitudes on a 5-point scale. The authors foiJIfd 
that in some ca3es, a lesson might get high ratings from 

.students. But* when they obs^ved th^ students in^jfehe class- 
room they had dif f icyrl ties in a few places. The director 
decided to add a note file, and urged students to leave 
comments about specific, problems they encountered. The 
authors^ found these were extremely valuable for revising 
trouble spots . Lessons could thus be , brought to a highly 
polished form^ -\ ' • . 

Two other projects decided to U3je note files because 
they were interested in, student reactions. Neither one of 
them found the notes par ti<i.ylar.ly ^useful fpr lesson revi- 
sion. Some 'comments were iti poor taste. The director of 

"yet another project decided that notes would be-tfseful. 
He anticipated that if many notes were sent frcp a par^ti- 
cular place in the l^esson,' it would indicate that was a 
poifnt of high frustration. However, the notes were too gen- 
eral to be ^of value in revising lessons. They were, fre- 
qtfcently derisive and discqsysed the instructor or the CB 
sys-tem. One corit^eflit ion common to several cfbarses whose , 
files became personally vitriolic, rather than idstructio 
ally oriented , 'was that in these nofrefiles students were 
permitted to read and respond to other students* notes'. 
Some instructors found it useful to provide a separate pl^e 

.for comments which were unrelated to the lessons. ^ * 

» » * ' 

3. Shift in procedure ' > 

Case'i4B3.1 - ShoVtcut evaluation procedure 

In three different situations, the. authoj^-director went 
through the following steps as part of formative evaluation 
(with revisions' as nec'essary): Review by colleagues, trial 
with a few students, trial with small classes, and finally 
actual use-with la^ge' cl-asses . As authors became more 
confident in -their skill dt Itesson production, they^vdec iSled 
that the middle step*s were not providing enough -additional 
information relative J:o the time Invested. EN^entually they 
decided to have a colleaigue review for cofrtent errors and 
then pat the lesson out for regular class use. 

The above sequence Seemed to be a common element^ of the 
development of design skills in a specific sub jec t-rfiatter 
area for a fixed garget populatioa. -Jt'was frequently - 
reported by experienced CBE authors in interviews/ The main 
danger was that the person assumed that skills iMrned in 
these specific circumstances apply universally. For* 
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example, attempt t^ ^rovidj^. fey^es of "fjersomalize'^^ 
f eadback/tbat ^as' found very 'effective for ^oung chiidren^ 
fawed cojnpletely for adult students who r^portid that the 
feedback wasted their time and was "chilxliah". 

. C Summative Data 4° ' • ■ > 

•1. '.Reports -to stu<ient-s* , . ' 

..Case 4C1.1 - Report prog?^||^ , ' ^ , 

One university prof.essor supplemented his regular 
course wi'th CBE l^s^ofiZ. 'He found that mbst o^ the student « 
who. rtiade* app^oin tments -^to. se^ him just "waKled to know how^ ' 
they were doing *4n the co\a|{|Se. He decided to' add -ai 
capability that enabled each student to s^' his .own 

^ ^l^w he had done .an - all • l^sojis arnl tests,'*c 

p^ed to th^e rest of th^^ class. * The result "^gs^^^-^ffam a 1 
drop in the number of ^students who tfame' to see him.^ufling 
office hours and indreaSed reports of satisf act ioYi with the 
^ course. ' ' ^ ^ . . . »^ ^ - 

•> • ' * ' . V. • w , - \ ' • \ 

Case' 4C1.2 - Failure , to rebort status ' ■ . 

r . ,^ . \ - 

^ One project developed a o^w' curr iculum^^for CBE but did/ 
not include any method of rep.o'rting %m the studen^ts -how far* 
they had^come br where' they stood in ^he cQlir.sa. This defi- 
ciency, not knowirtg where they were at/^v^as a major • com>/^ \ 
plaint of students whd protested^ against being in , the CBE 
sections. V , \ ^ / * 

2. Me assures u^wd or "needed^ .* . . ^ ^ 

Case ^C2.1 -iB^ar^ting jout specific CBE effects* j J 

• * . *^ ^ , 

Many groups u§^ed final performance ' sc6^es to' cbmpare 
CBE with contr.oi students. This was.^o't a jsat isf actory mea- 
sure because CfeE effects <Jere eonfoundecJ with other effe(<tsr^ 
^ 'Consequently some eValu^tora found significant tii^f f ereryces , 

butp. many* did not. Author^ in one group d.ecided 4p write • 
, their* .o,wn ;c^terion-referea?ed test^ to compsire CBE instruc- 

rtion v^ith noti-CBE" instruction . on. ^ set of lessoes wi a ^spe- 
cific I topic* ~ \ ^ ' ^. , \ . ; 

f The 'results showed^ that- students completecU almost afll . 
of, th# les4€n|| but the time tliey^'took varied considerably. 
Thjperel were* rto,. perf ormance differences betw.een CBE 'and noh^ 
CB^;.*' s|fcudents\ ; However /* tHe authors used specific item 
, inTomation t6^ imprpve thj^less'ons . , * , ^ . 
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Ca'se\4C2.2 - Evaluati». shpuld iijjclucle information" *about 
studentgf IJtvels " \ 

. JDne grouf^Tbllect^d time and p^erformance dal 
abil^ity-level .students during student trials of J 
sons'. ' The sar^e lessons y^ere, then tried by/^-difl 
higher level • )6tutfent*group. Data revealW ceiling' eff ects; 
the material Vas ta# easy for. the second gro'up, 'so the 
authors decided; to upgrade the level-- of the materials. They 
tried the* lessons afi(a thii;d student population and. failed 
to achieve, successf uKper formance . The authors concluded/ 
that %he lessons needed revision again to^eet the needs of 
this group..' ^ - 

Case'^C2.3 - Record of time spent in les^ojj^K 

^--vjteny ^projects kept a record of tim'e student s spent in .a 
lessottff It is. not clear in which cases it was the result of 
a conscious decision, and iz\ which the time was kept because 
it was easy to obtain. The uses of tlT^ time .data varied. 
Time spent in lesson, provided useful indicatJ.ons of the need 
for revisions. If students needed: ipore th^n the allotted ♦ 
time to complete a lesson, tri% author surmised it was too 
'h^ard and/or too long. If a student • scorecj poorly on a 
post-test but^ad spent considerably less time than average 
l^iu .the le^sonj 'the author assuirfed that the failur.e was not ' 
the,,qual^y of the lesson, but the failjjre of the studljnt to 
dom^l^^ it. • , . . ^ • . . \ , ' 

3 . -Post-tests ' * , 

Case 4C'3l^ - Burden of evaluation, 

In one case ^ JLes^ons Vere written at a curriculu 
development centefV^ Instructors at the. coopera'ting i 
tution refused to give post^tests cl^aiming it* took tci#' much 
^extra^ime. . ^ ' 

instructors at andther institution gave post<-tests 
before ^students Had compjL-eted the CBE lessons'. Others 
waited until itlwas very iate in the semester , and. students 
"^wer^ too busy -Co take thtem. Resul^: about 20% of^ the stu- 
dents took the' post-test and this was Nlnadeq^uate ba('9is / 
for evaluation., , * / • ^ \ 

In other cases, instructors or the in^i tution were * 
paid for the Work entailed in administer*ing >ex,tra tests.. 




Need to monitor data collection 

C^se '404^1 - The need to mdnltor jlata collection ' ' 

At th»e request of an external evaluator, data were k^pt 
',6n the time students spei^t in lessons'^ in one project.' The 
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eyaluators ant^nded to* collect data for a specific tflrass 
rather ^thai^'for -all of the stifdents vfho c^id the lessons^ *^ 

.Moreover tteyv intended to delete the'data of those" student's 
who dropped out of the clas5 durin.g the year because, such 

^subjects would n5t be ayailble for posttests . - T+ie data 
gathering pn&|fram was .written 'and/data were .collected . No 
individual was assigned the responsibility for.'rt^ After ; • 
the proje,ct was completed, it was discoveV-ed that the ' 
program was collecting data on all students, and not limited 
to thos6 in tne study. Fur thvermore , the program for • 
drol)ping studenlkp' was deleting student's ' from the end of the^ 
list rather tha,n those who had dropped out.* Pf 'someone had 
6een specifically assigned to monitor .the data • colTiPttion, 
these errors could have been corrected early'in the ^project. 

Case - Tnappropr i-ate>Tests ^ 

/ ' An outside evaluator'" dWised test it>ema to me&sure* ' 
the achievement of students /^al<ing a set of C3E lessons-. 
Author^ reviewed the^ items and found them to be insensitive 
to the T>bjectives of the lessons.^ Authors felt that the 
outsiders we're,' in fact, lacking familiarity with the CBE' 
system and' its impact on the ^students. 

In another project, the CBE instructor .agreed to use " 
-the final test written by the feraditionral ctassroom 
instructor, for a compar ison « of media. Forms pf the same 
tesjt had been u^ed for several years, ^and the CBE instructor 
assumed, that iSrior tests d»*i^necl the desired co'ufse - 
objectives {which the GBF Tnaterial had been designed to 
teach). However, the traditional classrc^oB^ instructor radi- 
cally altered both test and obj ec-t iVe's^S^t the last minute. . 
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' Chapter 5 • 
MAINTENANCE:' 



Lesaons needf to'-^be maintainecf-and plasfsrooms should be * 
monitored even af If^erj lesson^ have beet 'successfully used for 
many classes. A humber of reasons support this, contention'; . 
( 1 ). Informat^ion^msfy need' to be updated. (2) New program ' ' 
errors may occur^ (3)^etter ways of presenting- the nja.te-^i' 
rial ;may become' apparen^. ^ (M-) On-line *fla.ta dontt ttfll * ( . 
everything about what students were doing in th6* classroom. 
ADd,» finairy-K (5) characteristics* of the target popuIa^i5n 
may change. Critical incidents ^are ^ot available to support . 
e\rery one of these reasons, "but experH(fc^ "Is;. ^accumulating , 
to suggest .thaVthey are qul^e re^listi^ / 1 ' 
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A. Classroom "Management ^ ^ 



1 . Proctors 



iVr ftp 




Case 5A1.1 - Sm:Qpth§^r ftplementation ' ' ^ ' ' ' . * 

g Students ih one coijrse weVe sometimes irritated a(t 
particular ooint^ In T.€?sSons arid wi^t4 very negative com- 4 
ments. The* dir'eciio'r decided to hatre a proctor available • 
during, class time. Ap a result, ^tuden-ts tended to discuss ' 
problems with-'the* pr^tpr. Whene^ver such a proctor wa(4 
available, the typed commeifts of students were consistently 
less negative. Irl^ addition , proctors could keep authors . 
posted on new errors which had previousl^^ gone uhdiscove^ed 



Gas 



^5A1.2'^r Pt^event probLems 



One profwsoj Was short of funds-, sjx^h^ did not. hire a, 
proctor for hj^s €BE classroom.' lis a result, some stjJdents'^ 
signed in und^r more, than one name. Essentially they took, 
away, the learning privileges .of others. +1any tcTrminals^ 
needed adjustment for using microfiche. Students needed • 
instruction on l\ow to do this. .The professor decided to 
hire a f/roctor, It ledst-^ for t>ie first few weeks of the * 
semester i . ' ' . 



Case 5Al;3 -;Ne^d |br skillful classroom proctor 



OA wa^ tn charge af ^ 
I** I ' 



' In one^ cas^f-^n unquayrfied.v per.s 
classroom^ of CBt terminals. This singlBi clasf^roort wa.ift 
intended to* be available to students from ma!gY courS^^ 
Unfortunlat'lly, - the classroom- was over scheddled . .At the'^ 
beginning gf^the semester*, the rtumber of; students ^Vfee^eded 
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'the number of terminals. Instead of explaining the problem 
to tW students and attempting to maW some ^Dther arrange- 
ments, the classnoorrt manager was rude aiVd^ antagonlstte. ''As 
a r^^sult, man^ students ^jnply- refused to return to use the 
terminals^ even whe^* the scheduling problem was solved*. * 

C^se 5AJ .4 Shifting role o? proc^ 



One departmeiit decided , on the advice of an evaluator, 
to-Jia^ve a prootor in* , the terminal classroom on a regular 
basTfe. . When lessons were Jli^£S^*^sed for regular classes,; 
some of ^the, lesson^' were xnown to have problem^s, such as 
ina'd%c)uate iarrsw"^ judging ' at4 specif ic 'points . The prpctors 
Were able to help st4Jdents to g^t through "trouble spots and 
corriplete. the le^ssons. \ ^ . * - * \» . 

• 4CteV leslsoiJ re^visi5ns and several semesters of -experi- 
*ence), these problem 'spots* were ironed^-out aiid the proctors^ 
^ rol^ after^ "the firsji few w*eeks>-became oite of helping sty-* 
. ' dents witb .off-line problafns. Other rol^s of proctors were 
*^ to -write probl^ems, tq ' send' notes; to. leSson authors, land to 
encourage students to write conjment^. * 'J, 



Chapter 6 ^- • \ 

SUMMARY ' ' 

Four major fScftors that helged PLATO , pro jects succeed 
were: attention to develop^infe good relations "with instruc- 
tors and administrators , clear goal specification , early " ' 

Splans/for lessen validation;, and evaluation, and plans for 
contingencies 'or al te*r;;atlons when circumstances necessitated 
^deviations from initial plan§. » i 
' Staffing was p^ticularly difficult in the early years, 

"because nobody knew whicR* related qualifijcattpn^ were mpst 
imparta^jt for authors. Almost nobotly had had experience in 
CBE. It turned out that successfiul authors Were peaple 
whQse qualifications eventual ly* included teaching, instruc- 
tional design rkrtowledge, content expertise, and knowledge of 
the TUTOR lap^uage. In the early years, authors had to 

learn TUTOK ^without printJed handbooks or references , ' because, 
the system was changing too rapidly to keep printed material 
, up-to-date. They were therefore particularly in need of 
. psychologfcal support from a "master" a'ut%)r or consultant - 
to serve as a model, to help, them over specif i? problems as.' 
they ^ro^e, and to guide them *as to which subset . of Aher 
TUTOR language to. learn first. ^ ' - 

There was no single best model of staff organization 
' for PLATO l%ssbn development. Some groups ^worked as inde- 
pendent author^ and others were organized as teams of var- 
ious sizes. An advantage of the* independent author over the 
teams was that the author did not have the problem of tpyijig 
to communica»te his ideas to another .person for implementa-^ 
*tion; he could execute the lesson exactly as he chose. He* 
could create and revise as he "developed the lesson and was 
not constrained by- the necessity of prespecif icBtion of 
jdetails for Somebody else., Tj^is worked best for 'authors who 
were experienced teachers and who l)ecajp^ proficient pro- 
grammers. /A d-isad.v^ntage ,of independent authprin^ was the 
variable quality of les'sons that Resulted * in some cases.' 
Authors' bTOAme very ^go-inVolved* and resisted making cfianges 
- "to" impr'apeM^ssons . .Th? team approach did not engeoder such 
ego invoJ^ir^^ Since, not all' teachers lor content e-xp^ts 
were proirilP'ient programmers^ the' tj^m Approach enabled them 
to concfntrate on content and' -i'^s^uctioni^ matters and 
leav& the coding to'others. The most successful teams wel^e-S 
those in which each m^ember of the team knew something about 
th^ oth^r members' domains. \ A drawback .to the 4?eam approach 
was tha^t* irff^ases where a team was put together wlthouV 
clear leader;5hip, internal fighting and power s\ruggljs 
, often occurred with resultay^t loss of produ'ctivi^y . ^ - 
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.-Formative evalua.ti^ was a necessary part of lesson 
development; and iserved as $r basts for lesson revision. 
Failure to' plan/for student trials often resulted^ in gross . 
programming erjrors, unuseable lessons, and frustrated stu- 
dents. Each kind of evaluation (lesson reviews, student 
rials, on^-lfTTf da ta-'^o^l lection) provided di-fferent Kinds of 

ofje could not sery.€L_5S a substitute for the 
alidation. and data on effectiveness could-, 
y when sumnfativ«e 'evaluation had beerLjare- 



informatioh , a 
other . ' Lesson 

only be 'obVainera wnen summaciv«e 'evaluation had Deer^w^^ 
plaaned^.Snd nefcessary data were collected' and ^oni^Pwd, 
1 In)tegra tiKxn with Obther instru^ition required special 
a ttfenttfon to i\nvolvement .of instructors in a jneaningful \ray. 
Cl'assrpom implementation required constant monitoring for 
overall mSriagem^nt as well ag lesson maintenance. 
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I GLOSSARY - . . _ 

authfor:* aff individual who developed and executed all 
'« aspects" of a CBE lesson: content, instructional design, 
display forroa-ts, programming. ' . 

" CBE; computer-based education; instruction del'iv^ered by 
compute. c - . y 

CERL: Computer-based Education Research- Laboratory, head- 
quarters of the University of Illinois PLATO system. 

N 

director: the person who wasv responsible for, a CBE project 
• or site . " 
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electFoni.c page turner: a |pE lessbn which is essentially 
like a t^Tttbook.' It is a sfccessio.n of. displays which pro- 
vide li^le or no interaction between the- student and the 
computer. 's, ' ' . 

external source: an institution where lessons^were deve 
oped, separate fro"m the school which eiirolled the target 
population. . , . # 

instructor*: individual who teSch^s traditional material and 
*llso (possibly) uSes CBE lessons. \ An instructor might also 
be an author .of CBE lessens. , 

on-site ^onsultantx an indivitlual who was at the same phys- 
ical locatiorf as the au^thor and provid'e^help to overcome 
^rogratffming proolems. • * . ' , 

• ' ' : ¥ 

sigridn: an Identification given to a person whiSi^ enables' 
him to work at the CBE- terminal . * ' , V 

sit^*: .a project at a gt^vtn geographical location or a pro- 
• ject developi.n&» cu'rridulum for a^ particular; subject. 

•TUTOR; - th'e special prbgramming fangniage of the PLATO 
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